Inter-operating system storage services connected via a communication network

Abstract

Systems, methods, and apparatus for providing storage as a service are described. For example, a tenant device is configured to: communicate with a lessor device; borrow an amount of memory from the lessor device; expand the memory capacity of the tenant device for an application running on the tenant device using at least the local memory of the tenant device and the amount of memory borrowed from the lessor device; and provide services for access to memory by the application over a communication link between the tenant device and the lessor device.

Description

[0001] Related Applications

[0002] This application claims priority to U.S. patent application serial number 16 / 424,411, filed on May 28, 2019, and entitled “Inter-operating system memory service connected via a communication network,” the entire disclosure of which is hereby incorporated by reference herein. Technical Field

[0003] At least some embodiments disclosed herein relate to memory services provided by an operating system. Background Art

[0004] In some computer systems, an operating system allows an application to use a virtual address of a memory to store data in or retrieve data from a memory component of one or more memory subsystems of a computer system. The operating system defines a mapping between a virtual memory address and a physical memory address of one or more memory subsystems of a central processing unit (CPU) and / or a graphics processing unit (GPU) and / or a system on a chip (SoC) connected to the computer system. Such a mapping can be defined using a page table. A page table entry specifies a mapping between a set of virtual memory addresses of a virtual memory page and a corresponding set of physical memory addresses of a physical memory page. A page table can be used to implement memory access for a virtual memory address request by translating the virtual memory address into a corresponding physical memory address of a memory subsystem.

[0005] A computer processor may have a memory management unit (MMU) configured to perform translation of virtual memory addresses into physical memory addresses according to a page table defined by an operating system. The memory management unit (MMU) may have a translation lookaside buffer (TLB) configured to cache recently used page table entries. The memory management unit (MMU) may perform other memory-related tasks, such as cache control and / or memory protection.

[0006] Typically, a virtual address space for memory in a computer can be divided into pages of a predetermined size. A page of virtual memory is represented by a collection of virtual memory addresses; a virtual memory address can be mapped to a physical memory address of a page of physical memory in a memory subsystem; a page table entry defines an address mapping between a page of virtual memory and a page of physical memory.

[0007] In some computer systems, an operating system may use paging techniques to access pages of data in a storage device or memory device through pages of memory in a memory module. At different instances in time, the same page of memory in a memory module may be used as a proxy to access different pages of memory in a storage device or memory device or another storage device or memory device in the computer system.

[0008] Remote Direct Memory Access (RDMA) is a technology that allows direct memory access from one computer to another without involving any of the operating systems of the computers involved (e.g., after the operating system is executed to negotiate and set up memory resources for RDMA operations). Prior to an RDMA operation (e.g., read or write), commands are executed to establish a memory mapping between the two computers through one or more networking devices. When an application running on a computer performs an RDMA operation, the application data is passed directly over the computer network, thereby reducing latency and enabling fast data transfer. RDMA uses a network adapter to transfer data to or from application memory, eliminating the need to copy data between application memory and data buffers in the operating system. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Embodiments are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings in which like references indicate similar elements.

[0010] Figure 1 A system is shown in which a computing device or server computer can borrow memory from another computing device and / or another server computer and / or rent memory to another computing device and / or another server computer.

[0011] Figure 2 A tenant device and a lessor device implementing an inter-operating system storage service connected via a communication network are shown according to one embodiment.

[0012] Figure 3 A technique for using borrowed memory according to one embodiment is presented.

[0013] Figure 4 The use of a physical memory region to facilitate access to a different borrowed memory region according to one embodiment is demonstrated.

[0014] Figure 5 Another use of a physical memory region to facilitate access to a different borrowed memory region according to one embodiment is presented.

[0015] Figure 6A tenant-lessor memory hierarchy in a computing system is presented.

[0016] Figure 7 A method for implementing inter-operating system memory services over a communications network connection is shown.

[0017] Figure 8 A method for servicing pages of borrowed memory according to one embodiment is shown. DETAILED DESCRIPTION

[0018] At least some embodiments disclosed herein provide technology for inter-operating system memory services connected via a communication network, such that a lessee device can expand the memory capacity of the lessee device by using the communication link and the memory of the lessor device. The lessee device and / or the operating system of the lessor device can be configured to seamlessly bridge the gap over the communication link, such that applications running on the lessee device can indiscriminately use the local memory and borrowed memory of the lessee device. The local memory is physically installed in the lessee device; and the borrowed memory is physically installed in the lessor device connected to the lessee device via the communication connection.

[0019] Optionally, a portion of the borrowed memory and / or local memory in the lessee device can be loaned to another device over a computer network. A lessee device can have multiple lessor devices. Thus, a lessee-lessor device hierarchy can be formed on a hierarchy of communication links between devices.

[0020] The operating system of the tenant device can use a mapping of virtual memory to physical memory to bridge the difference between local memory and borrowed memory. For example, the operating system of the tenant device can manipulate a page table entry (PTE) in response to a mishit of a determination in a translation lookaside buffer (TLB) of the tenant device that may cause a page fault to use a portion of the local memory of the tenant device as an aperture, proxy server, and / or cache when accessing the borrowed memory over a communication link. Cache coherency between the lessor device and the tenant device can be relaxed to improve bandwidth utilization. Because the difference between the local memory area and the borrowed memory area is shielded from interference by applications running on the tenant device by the operating system of the tenant device, applications running on the tenant device can access the borrowed memory byte by byte in the same manner as accessing the local random access memory configured in the tenant device.

[0021] The memory services of the operating system can be used to build multi-level byte-addressable memory in a computer system under a unified address space shared by multiple operating systems through non-coherent communication links. For example, a wearable computing device can borrow memory from a mobile computing device through a personal area network or a local area network. The wearable computing device can be a smart watch, a sports tracking device, or smart glasses, and the mobile computing device can be a smart phone, a touch pad computer, or a notebook computer, etc. The wearable computing device and / or the mobile computing device can borrow memory from a computing device through a local area network. For example, the computing device can be a pocket server, a desktop computer, a memory device, or a storage device, etc. The mobile computing device and / or the computing device can borrow memory from a server computer or a server farm through the Internet and / or through cloud computing.

[0022] In a multi-level hierarchy of lessors-lessees of memory resources, each subsequent higher-level device (e.g., mobile computing device, computing device, server computer, or server farm) can use its memory capacity to increase the memory capacity of lower-level devices (e.g., wearable computing device, mobile computing device, computing device) and / or devices of the same level as needed. IoT devices will form the environment, like building materials (concrete walls, roads, etc.). Cheap memory technology, which may be low-density but bulky, will allow low-density memory (and computing) to be integrated into the construction at very low cost. Therefore, some kind of "memory substance" (or perhaps computing substance) will be made part of the environment that any device can use.

[0023] Optionally, certain levels in the hierarchy may be skipped, depending on the availability of certain devices and / or network connections. For example, a wearable computing device may borrow memory from a computing device without going through a mobile computing device; and a mobile computing device may borrow memory from a server computer or server farm over the Internet without going through a computing device in a local area network.

[0024] At the device level and the cloud level, memory resources can be provided as a utility with memory as a service (MaaS). Such an arrangement can provide low-level computing devices, such as wearable computing devices, mobile computing devices, and / or computing devices of the Internet of Things, with huge persistent memory capacity and storage resources on demand. Applications running on the low-level computing devices can use the memory services provided by the operating system to use the borrowed memory in a transparent manner, just as if the borrowed memory was part of the local memory of the lessee device. Therefore, no special programming is required in the application to use the borrowed memory.

[0025] Applications running on the tenant device may use the borrowed memory to persistently store data and thus eliminate or reduce the need for operations for storing data through the file system. For example, the lessor device / server may provide persistent random access memory or storage capacity to provide memory resources to the tenant device and manage the persistence and / or redundancy of data stored in the provided memory resources, so that the borrowed memory may be viewed as a virtual non-volatile random access memory in the tenant device.

[0026] Optionally, the storage provided as a service may include semantics for memory page sharing between clients / tenants, locking mechanisms, timeout mechanisms, etc.

[0027] As an example, a phone may obtain 8 terabytes (TB) of observable memory from a memory device acting as a lessor device over a wireless local area network (WLAN) (e.g., facilitated by a home Wi-Fi access point) as a lessee device. Applications running under the phone's operating system may observe the 8 TB of memory in the same manner as applications running on a phone with 8 TB of local memory. Alternatively, the phone may obtain the 8 TB of observable memory from a server or cloud computing infrastructure configured as a lessor device over a cellular communications network. Optionally, the phone may borrow a portion of the 8 TB of observable memory from the memory device and may borrow a portion of the 8 TB of observable memory from a server or cloud computing infrastructure. Even though the phone may only have 8 megabytes (MB) of physical local memory, the 8 TB of observable memory may be made virtually available on the phone through the phone's operating system so that applications running on the phone can use the virtual / remote memory. The 8 TB of borrowed memory expands the phone's memory capacity to more than 8 TB; and to applications running on a phone with borrowed memory, the phone's memory capacity appears to be the same as the memory capacity of a phone with more than 8 TB of local random access memory.

[0028] Since the lessee device typically does not use the borrowed memory immediately when borrowing the memory, the lessor device does not have to block allocation of memory loaned to the lessee device while the memory is borrowed. The lessor device can implement thin provisioning of memory (e.g., thinly provisioned memory) and incrementally allocate memory using increasing amounts of borrowed memory in response to the lessee device.

[0029] Applications running under the operating system of the tenant device can read and write memory pages borrowed from the lessor device on demand. Physical page allocations in the lessor device and / or corresponding allocations in the tenant device can be configured to be executed in response to write operations to the page. A portion of the local memory of the tenant device can be configured as a cache, a proxy server, and / or a hole when accessing the borrowed page. When an application running on the tenant device accesses a virtual page, the operating system of the tenant device can migrate the content of the accessed virtual page to the local memory. When the local memory page has not been accessed for a period of time and / or is to be used to host the content of another page, the operating system of the tenant device can evict or migrate the content of the virtual page in the local memory to the borrowed memory on the lessor device. Optionally, the operating system of the tenant device can use a portion of the local random access memory of the tenant device as a fast buffer for critical data, and use the borrowed memory physically residing in the remote lessor device to keep less important data. When necessary, less important data and / or less frequently used data can be migrated to the local random access memory of the tenant device.

[0030] Typically, a lessor device of a lower level device in the hierarchy may itself be a tenant device of a higher level in the tenant-lessor device hierarchy. For example, a phone as a tenant device may borrow 8TB of memory from a memory device as a lessor device. The memory device as a tenant device may borrow 4TB of memory from a server, and the 4TB of memory borrowed from the server and the 4TB of local memory of the memory device may be combined into the 8TB of memory loaned to the phone. In some embodiments, the 4TB of memory borrowed from the server may be identified by the phone so that when the connection between the phone and the device is unavailable, the phone may continue to use the 4TB of memory without going through the memory device using an alternative connection to the server. However, because the memory device may be closer to the phone than the remote server, the alternative connection may be slower than the connection to the device.

[0031] The operating system of the lessor device and the operating system of the lessee device can communicate with each other to facilitate seamless memory access across the communication link between the lessor device and the lessee device. Therefore, it is not necessary to pre-allocate physical memory in the lessor device before the application running on the lessee device begins to use the memory. As the footprint of the application running on the lessee device increases, the page eviction traffic and the lessor device memory availability determine the size of the borrowed memory usage. Eliminating the need to pre-allocate physical memory in the lessor device for the application (e.g., as required in the case of the RDMA protocol) eliminates the need for specialized programming (e.g., through the message passing interface (MPI) of RDMA) and allows the application to run in devices of different configurations with the original code of the device, but just as the device does in the case of RDMA, etc., with all the benefits of scaled-out application capacity.

[0032] Typically, a lessor device may be configured to operate in a passive mode or an active mode with respect to a particular lessee device. A lessor device may operate in different modes with respect to different lessee devices.

[0033] In passive mode, the operating system of the lessor device observes memory pages that are cached or migrated by the operating system of the lessee device through the link and modifies the operating system's own page table entries corresponding to these pages accordingly. For example, the observation of page movement by the lessor operating system can be performed through bus snooping or hardware monitoring.

[0034] The operating system of the tenant device is configured to maintain page tables for the local memory of the tenant device and the borrowed memory physically residing in the lessor device. Typically, when a page of the borrowed memory is being accessed by an application running on the tenant device, the contents of the page may or may not be currently stored in a page of the local memory of the tenant device.

[0035] A page fault occurs when a tenant device accesses a page that does not exist in the tenant device's local memory. For purposes hereof, a page fault is considered to occur in two cases: 1) the accessed page does not exist at all; and 2) the accessed page exists but is not in local memory.

[0036] If the page does not exist, then the page has never been written or read. There is no page table entry (PTE) for the page. This is the first time the page has been accessed. If a read operation is performed on such a page, there is no need to allocate a real page. Some operating systems (e.g., Linux) can provide a dummy page filled with zeros because the page has never been written and therefore does not contain any data. Therefore, the result of a read operation on such a page is zero. In this case, the OS will execute a handler for this page fault: a TLB miss will cause a page walk, which determines that a read operation was performed on a page without a PTE. Therefore, a mapping to the zero page is returned.

[0037] If the first write operation is performed on the page, the page is truly faulted. A true fault results in a new PTE in local memory and page allocation. Example of a page fault handler for this case: A TLB miss results in a page walk, which determines that a write operation was performed on a page that does not have a PTE. Therefore, the operating system allocates a new page, generates a PTE, and installs the PTE in the page table. After the PTE is cached in the TLB, the fault is resolved.

[0038] When a page fault occurs when accessing a borrowed page and the contents of the page are not currently in the local memory of the tenant device (e.g., after the contents of the borrowed page have been migrated / evicted to the lessor device), the operating system of the tenant device can communicate with the lessor device via a communication link between the tenant device and the lessor device to retrieve or migrate the contents of the borrowed page.

[0039] The cache operation can be performed in conjunction with the processing in the two cases discussed above. For example, the lessor device can retain a copy of the page. The copy may be useful if the lessee device does not further modify the page (e.g., only performs read operations on the page). In this case, no eviction is required because the true copy of the page already exists on the lessor device. Similarly, the lessee device can perform speculative evictions of pages when the communication link between the lessor device and the lessor device is free for use.

[0040] Optionally, the lessor device and / or the lessee device may be configured with firmware / hardware that accelerates the operation of migrating content between the lessor device and the lessee device.

[0041] Optionally, the operating system of the lessor device may be configured to maintain and operate its own set of page tables and associated resources, such as page table walkers, translation lookaside buffers (TLBs), content addressable memories (CAMs), lookup tables, and the like.

[0042] The operating system of the lessor device monitors memory access requests from any of the tenant devices that borrowed memory from the lessor device. The operating system can be configured to provide semantics for page sharing between tenant devices (e.g., atomics, locks, semaphores, flags, timeouts, etc.). Thus, the address space of the borrowed memory is unified between tenant devices according to the page table in the lessor device. Distributed applications can utilize the unified address space by sharing the borrowed memory through the lessor device with different instances of the application running on different tenant devices.

[0043] Optionally, the lessor device can be configured to accept requests from the lessee device to generate computational results on certain areas of memory loaned to the lessee device. Thus, the lessor device provides not only memory resources but also computational capabilities for processing data in the memory loaned to the lessee device.

[0044] The memory services provided by the operating systems of the tenant and lessor devices allow applications running on the lessor device to address borrowed memory at byte granularity while faults are resolved at page granularity.

[0045] The operating system of the tenant device can evict or migrate pages from local memory to the lessor device via a communication link between the tenant device and the lessor device. The operating system of the tenant device can identify pages for eviction or migration from the local random access memory of the tenant device to the borrowed memory of the tenant device based on the following: the page is not frequently used and / or the page is less recently used than other pages or criticality or any other possible measurable system parameters. The operating system of the tenant device can decide to evict or migrate the borrowed page to free up local memory for other uses, such as for new local page allocations or for migration of pages from the lessor device. As a result of this approach, the current page owner (tenant or lessor) has a true copy of the page. Other devices may still cache the page in case the page remains unmodified after the owner releases the page. For example, when the tenant gets the page from the lessor, the lessor no longer has the page. However, the lessor may not delete the page copy, but retain the page copy: if the page is not modified when it is evicted, it will not be necessary to transfer the page back to the lessor over the link. What if the lessor also wants to modify the page or provide the page to other lessors? In this case, all writes must occur on the lessor in order to merge write requests from different lessees.

[0046] To evict or migrate a page, the operating system of the tenant device transmits the contents of the page to the lessor device, causing the lessor device to store the page in the memory of the lessor device. The operating system of the tenant device may then update the page table to indicate that the page is now located in the lessor device and is not located in the local memory of the tenant device.

[0047] In active mode, the operating system of the lessor device may be configured to: monitor page tables on the tenant device; identify unused pages; and repurpose the unused pages for other memory users.

[0048] The interaction between the lessor device and the tenant device may be implemented through virtualization, which allows the lessor device to monitor page table usage in the tenant device.

[0049] For example, the operating system of the lessee device may be partially virtualized for operations in the operating system of the lessor device. Some of the memory management code / subroutines of the operating system of the lessee device may be configured to run on the lessor device and / or be configured as part of the operating system of the lessor device through virtualization. In such an embodiment, the operating system of the lessee device may extend a memory access request to the operating system of the lessor device to execute such subroutines that may be hardware / firmware accelerated at the level of the operating system of the lessor device.

[0050] For example, the operating system of the lessor device can create a virtual memory device and provide the virtual memory device to the tenant device. The tenant device can request the virtual memory device on demand and make the virtual memory device available to serve the memory resource requirements of the application running on the tenant device.

[0051] In some instances, if the application running on the tenant device can tolerate or accept the delay caused by the communication link between the tenant device and the lessor device, access to the borrowed page can be made directly over the communication link without migrating the entire page to the lessor. In other embodiments, content migration can be configured at the sub-page level; and only a portion of the page being accessed and / or predicted to be accessed is migrated. Optionally, the operating system can use remote direct memory access (RDMA) technology when implementing direct memory access. Optionally, such direct memory access can be implemented at the memory controller and / or memory management unit, and thereby avoid the standard RDMA protocol and the overhead of the protocol.

[0052] In some embodiments, the operating system of the tenant device is configured to dynamically adjust the physical location of the virtual memory page. A page of virtual memory is specified by a virtual memory address of the page. The virtual memory page may be initially located in the local memory of the tenant device, subsequently moved to memory loaned to the tenant device by the lessor device, and subsequently moved to another local page of the tenant device.

[0053] To migrate a virtual page from a lessor device to a lessee device, the lessee device obtains the content of the page from the lessor device, stores the content in a local memory of the lessee device, and updates a related page table to establish a mapping between a local physical page and a virtual page. Once the virtual page is migrated to the lessee device, the lessor device may release the physical memory previously used for the virtual page and / or erase the data stored in the lessor device. Optionally, the operating system of the lessor device may retain the page previously used for the virtual page and retain the content of the page as a backup copy of the virtual page.

[0054] In the reverse direction, the operating system of the lessee device may migrate the virtual page from the lessee device to the lessor device. For example, the operating system of the lessee device may request that a borrowing page be allocated in the lessor device, and the contents of the virtual page be transferred to be stored in the borrowing memory on the lessor device. Once the contents of the virtual page are stored on the lessor device, the operating system of the lessee device may update the page table of the lessee device for the virtual page to identify the mapping between the virtual page and the physical location of the virtual page in the borrowing memory to complete the migration of the page to the lessor device. After migrating the virtual page, the local memory previously used to host the virtual page may be used to host another virtual page. In such an embodiment, it is not necessary to statically map the virtual memory or a predetermined portion of the virtual memory address space to the borrowing memory in the lessor device.

[0055] In addition, the virtual page does not have to be hosted in a predetermined lessor device. For example, the operating system of the lessee device can migrate the virtual page from a first lessor device to a lessee device, and then migrate the virtual page from the lessee device to a second lessor device. In some examples, the operating system of the lessee device can request that the page be migrated from the first lessor device to the second lessor device without downloading the content of the virtual page to the lessee device and then uploading this content to the second lessor device.

[0056] Using the technology of Memory as a Service (MaaS), mobile device vendors can sell mobile devices with terabytes of memory capacity at the operating system level, which will be supported by emerging memory on lessor devices including cloud computing infrastructure. For example, when a smart phone is connected to a computer network or the Internet or a local memory device or any other device implementing the MaaS concept described herein, an application in the phone configured with 128MB of local memory can access 64TB of random access memory.

[0057] Figure 1 A system is shown in which a computing device or server computer can borrow memory from another computing device and / or another server computer and / or rent memory to another computing device and / or another server computer.

[0058] Figure 1 An example of computing devices (101 and 103) and server computers (105 and 107) as lessee devices and lessor devices connected via a computer network and / or the Internet (109) is shown, wherein the computer network and / or the Internet may include a cellular communication network, such as a fifth generation cellular network.

[0059] Each computing device in the computing devices (101 and 103) and each server computer in the server computers (105 and 107) can borrow memory from another device and / or rent memory to another device. Each computing device in the computing devices (101 and 103) and each server computer in the server computers (105 and 107) can have one or more communication devices (e.g., 117, 137, 157, and 177) to establish one or more communication links with each other or another computing device or server computer in the system. Each computing device in the computing devices (101 and 103) and each server computer in the server computers (105 and 107) can have one or more processors (e.g., 115, 135, 155, and 175) configured to execute instructions such as instructions of an operating system (e.g., 113, 133, 153, and 173) and applications or programs running under the operating system.

[0060] Each of the computing devices (101 and 103) and each of the server computers (105 and 107) may have a local random access memory (e.g., 111, 131, 151, and 171) coupled (e.g., via a memory bus) to a corresponding processor (e.g., 115, 135, 155, and 175).

[0061] In some examples, device A ( 101 ) or device B ( 103 ) may be a wearable computing device, a computing device of the Internet of Things, a mobile computing device, etc.

[0062] In some examples, server P (105) or server Q (107) may be a pocket server, a personal computer, a memory device, a storage device, a network storage device, a server computer, a server farm, etc.

[0063] In one example, device A (101) can connect to device B (105), server P (105) and / or server Q (107) using a wired connection, a wireless personal area network (WPAN), a wireless local area network (WLAN) and / or a wireless wide area network (WWAN).

[0064] In another example, device B (105) may connect to server P (105) and / or server Q (107) using a wired connection, a wireless local area network (WLAN), and / or a wireless wide area network (WWAN).

[0065] In another example, server P (105) can be connected to server Q (107) using a wired connection, a local area network, a wireless local area network (WLAN), a wireless wide area network (WWAN), and / or the Internet.

[0066] Server P (105) can borrow storage from server Q (107) and rent out the borrowed storage and / or local storage to devices (101 and 103) (and / or similar devices). Device B (103) can borrow storage from server P (105) and / or server Q (107) and rent out the borrowed storage and / or local storage to device A (101) (and / or similar devices).

[0067] Typically, a computing device may lend memory to one or more lessee devices and borrow memory from one or more lessor devices.

[0068] The operating system (113, 133, 157, and 173) is configured to provide memory services to applications and programs so that the applications and programs can use the virtual memory without knowing the physical allocation of the virtual memory, so that this virtual memory can be allocated partially in the borrowed memory and partially in the local memory (111, 131, 151, and 171). The operating system and supporting hardware bear the burden of managing the virtual memory allocation for the applications and programs, which provides convenience for the applications and programs so that they do not have to have a dedicated code segment or designated instructions designed to handle the difference between the local memory allocation and the borrowed memory allocation.

[0069] In one example, a portion of the local memory (171) of the server Q (107) is loaned to the server P (105). The server P (105) loans a portion of the loaned memory and / or a portion of the local memory (151) of the server P to the device B (103), and the device B in turn loans a portion of the loaned memory of the device B and / or a portion of the local memory (131) of the device B to the device A (101). Therefore, the virtual memory allocated by the operating system (113) to the application running on the processor (115) of the device A (101) may reside partially in the local memory (111) of the device A (101), partially in the local memory (131) of the device B (103), partially in the local memory (151) of the server P (105), and / or partially in the local memory (171) of the server Q (107). The operating systems (113, 133, 157 and 173) are configured to cooperate with each other to provide memory services and / or physical allocation of virtual memory used by applications between local memories (e.g., 111, 131, 151 and / or 171) of each device, so that an application programmed to run on device A (101) and the local memory (111) of the device A can also run without modification when the virtual memory of the application is partially allocated in the borrowed memory (131, 151 and / or 171) loaned to device A (101) by device B (103), server P (105) and / or server Q (107).

[0070] Figure 2 A tenant device (201) and a lessor device (203) are shown implementing an inter-operating system storage service via a communication network connection, such as a wired or wireless connection (205) via a local area network, a wide area network and / or a cellular communication network such as a fifth generation cellular network.

[0071] exist Figure 2 In the example, the lessee device (201) borrows the storage from the lessor device (203); and the lessor device (203) lends the storage (202) to the lessee device (201). For example, the lessee device (201) may be Figure 1 The device A (101), device B (103) or server P (105) in the system; and the lessor device (203) may be Figure 1The device B (103), server P (105) or server Q (107) in the system. Further, the lessor device function can be implemented in the lessee device (201) so that the lessor device can lend the storage to another device. Similarly, the lessee device function can be implemented in the lessor device (203) so that the lessee device can borrow the storage from another device.

[0072] exist Figure 2 In the embodiment of the present invention, a tenant device (201) has a communication device (217), one or more processors (215) having a memory management unit (MMU) (216), and a local random access memory (211). In some embodiments, the processor (215) and the local memory (211) can be in separate computer chips and / or formed on separate integrated circuit dies, and in other embodiments, can be packaged in the same computer chip and / or formed on the same integrated circuit die (e.g., in a system on a chip (SoC)).

[0073] An operating system (213) of a tenant device (201) includes a memory service module (209) configured to maintain a memory map (207). The memory map (207) identifies a mapping between virtual memory and physical memory, where the physical memory may be located partially in a local memory (211) in the tenant device (201) and partially in a loaned memory (202) in the lessor device (203). The operating system (213) provides virtual memory services to one or more applications (e.g., 212) running on the tenant device (201). A processor (215) may run an application (212) by executing instructions of the application code (212). Application memory read and write instructions may use virtual memory. A memory management unit (MMU) (216) translates virtual memory addresses into physical memory addresses according to the memory map (207).

[0074] When the lessee device (201) borrows memory from the lessor device (203), the borrowed memory (204) is virtually located in the lessee device (201) and the borrowed memory is physically located in the lessor device (203) as the borrowed memory (202) in the local random access memory (231). Optionally, the lessor device (203) can configure a portion of the borrowed memory (202) or the entire borrowed memory in a storage device (234) coupled to a processor (235) of the lessor device (203) via a peripheral bus and / or a device controller. For example, the operating system (233) of the lessor device (203) can allocate a portion of the memory resources of the lessor device as virtual memory to the operating system (213) of the lessee device (201) in a manner similar to allocating memory resources as virtual memory to an application (e.g., 232) running on the lessor device (203). Thus, the borrowed memory (204) used by the operating system (213) of the lessee device (201) can be located in a unified virtual memory address space as virtual memory used by the application (232) and / or borrowed memory used by other lessees of the lessor device (203). Alternatively, the operating system (233) of the lessor device (203) can create a virtual memory component for the borrowed memory (204); and the operating system (213) of the lessee device (201) can make the virtual memory component in the lessee device (201) accessible. A virtual memory controller for the virtual memory component can be implemented in the operating system (213) to access the virtual memory component accessible to the lessee device (201). The MMU 216 can implement a hardware block to accelerate the virtual memory controller, and by doing so, the speed of virtual memory component access can be increased.

[0075] In some instances, the borrowed memory (204) is implicit, such that the borrowed memory is indistinguishable from the rest of the virtual memory of the application running on the tenant device 201. The distinction only becomes apparent when the virtual page address is translated into a physical address by reading the memory map 207, and at the time of the translation, the physical address appears to be located in the borrowed memory 202. In other instances, if the virtual memory belongs to the borrowed memory (204) provided by the lessor device (203), the operating system (213) stores an explicit identification of the portion of the virtual memory. In all cases, the operating system 213 configures the memory map (207) to access the lessor device (203) to read from and write to the virtual memory as the borrowed memory (202) physically located in the lessor device (203) or located in the lessee device (201) after migration to the device.

[0076] The memory mapping (207) may be implemented by a page table. A portion of the page table is cached in a memory management unit (MMU) (216), so that when an instruction executed in the processor (215) accesses a virtual address defined in a portion of the page table cached in the memory management unit (MMU) (216), the memory management unit (MMU) (216) translates the virtual address into a physical address. Successful address translation allows the processor (215) to continue memory access without executing code from the operating system (213).

[0077] When the memory management unit (MMU) (216) cannot successfully translate a virtual memory address using the page table cached in the memory management unit (MMU) (216), a page table entry cache miss is generated, which causes the processor (215) to execute instructions of the memory service (209) of the operating system (213) to resolve this cache miss. In some embodiments, the MMU contains a hardware IP that is architected to resolve page table entry cache misses in the cache of the MMU (216). This IP extracts the required page table entry from the page table stored in the physical memory 211 as part of the MMU. If the required entry does not exist, or if the required entry exists but the entry translates the virtual page address into an address in the physical loan memory 202, a page fault is generated for this page. This fault can be resolved directly in hardware by the MMU 216 and the communication device 217. Alternatively, this error may be resolved by the operating system 213 and memory services 209 in software assisted by the following hardware blocks: MMU 216 , processor 215 , communication device 217 .

[0078] When the memory service (209) or MMU 216 of the operating system (213) determines that the virtual memory address being accessed is located in the borrowed memory (204), the memory service (209) or MMU 216 can allocate a physical page of the local physical memory 211, communicate with the lessor device (203) through the communication devices 217 and 237 and the computer network connection 205 to migrate the page of the borrowed memory (204) from the physical borrowed memory 202 to the physical local memory 211, generate a page table entry that maps the virtual page to the allocated local physical page, and load the page table entry into the memory management unit (MMU) (216). Therefore, before the page fault, the virtual page of the borrowed memory (204) is physically located in the borrowed memory (202) in the lessor device (203). After the page fault is processed, the virtual page is located in the local memory (211). The contents of the pages in the loan memory 202 of the lender device 203 may still be saved for future use and for other purposes.

[0079] When there are no free local pages available for migration, the memory service (209) may evict the selected virtual page currently located in the local memory (211) to the lending memory (202). When the selected virtual page is evicted to the lessor device (203), the memory service (209) communicates with the lessor device (203) to transfer the content of the virtual page from the local memory (211) to the lending memory (202), and updates the memory map (207). After the virtual page is evicted from the local memory (211), the space in the local memory (211) for the evicted virtual page may be released.

[0080] For example, the memory service (209) may be configured to evict the least frequently used virtual page or the least recently used virtual page.

[0081] exist Figure 2 In the embodiment, the lessor device (203) has a communication device (237), one or more processors (235) with a memory management unit (MMU) (236), and a local random access memory (231). Optionally, the lessor device (203) includes one or more storage devices (234) connected to the processor (235) via a peripheral bus and / or a computer network. For example, the storage device (234) can be a solid state drive (SSD) or a hard disk drive (HD).

[0082] The operating system (233) of the lessor device (203) includes a memory service module (229) configured to maintain a memory map (227). The memory map (227) identifies a mapping between virtual memory and physical memory. The memory map may be stored in the memory 231 or in a memory dedicated to the memory map, such as a content addressable memory or a dedicated cache. The operating system (233) provides virtual memory services to one or more applications (232) running on the lessor device (203) in a manner similar to the services provided by the lessee device (201) to the applications (212) of the lessee device.

[0083] Optionally, the lessor device (203) provides the loaned memory (202) to the lessee device (201) in a unified virtual memory space. The memory service (209) of the lessee device (201) uses the borrowed memory (204) in the same manner as the application (232) uses the virtual memory provided by the memory service (229). For example, the memory service (229) of the lessor device (203) can allocate virtual memory to the application (232) running on the lessor device (203), and allocate the virtual memory as the borrowed memory (204) used by the memory service (209) of the lessee device (201), just as if the memory service (209) was an application running on the lessor device. Therefore, the application (232) running on the lessor device (203) and the lessee device (201) (and other devices that borrow memory from the lessor device (203)) can operate in a unified virtual address space. Collaborative computing can be configured based on the unified virtual address space. Further, when the lender device (203) borrows memory from another device / server (eg, 105 or 107), the borrowed memory may also be provided in a unified virtual address space. A unified address space may facilitate data sharing and collaborative computing between computing devices.

[0084] The memory service (229) of the lessor device (203) can maintain a memory map (227) that includes a mapping between a virtual address of the loan memory (202) corresponding to the borrowed memory (204) accessed by the lessee device (201) and a physical address of the loan memory (202). For example, a page table entry for a virtual page identified by the lessee device (201) when accessing a borrowed page can be loaded into a memory management unit (MMU) (236) to facilitate translation between the address of the virtual page and the physical address of the loan memory (202) for a processor of the lessor device (203) to perform computations on behalf of the lessee device (201). Optionally, the lessor device (203) may include hardware similar to a memory management unit (236) to facilitate optimization and / or acceleration of data transfers between the lessor device (203) and the lessee device (201) using a memory map (227) that identifies a mapping between borrowed memory (204) known to the lessee device (201) and physically accessible loaned memory (202) in the lessor device (203).

[0085] Figure 3 A technique for implementing borrowed memory according to one embodiment is shown. For example, Figure 3 The technology can be Figure 2 The method is implemented in the tenant device (201) shown in FIG.

[0086] Figure 3 Shown are a memory (211), a memory management unit (MMU) (216) configured to access the memory (211) using a physical address (e.g., 257), and a memory map (207) defining a mapping between virtual address regions (e.g., 261, 263, 265, ...) and address regions to which the virtual memory regions are mapped, such as physical address regions (e.g., 281, 283, ...) and borrowed memory regions (e.g., 273, 275, ...).

[0087] For example, the borrowed memory address region (eg, 273, ..., 275) may be configured to identify a virtual address region allocated by the lessor device (203) as memory provided / loaned to the lessee device (201).

[0088] For example, virtual address region A (261) is associated with physical address region (281) in memory map (207) to indicate that the region of virtual memory is currently mapped directly to a corresponding region of local physical memory (211).

[0089] For example, virtual address region B (263) is associated with borrowed memory address region X (273) and physical address region S (283) in memory mapping (207) to indicate that virtual address region B (263) is mapped to borrowed memory address region X (273) and physically resides in physical address region S (283). Therefore, access to virtual address region B (263) can be achieved by accessing physical address region S (283) in local memory (211). Virtual address region C (265) is associated with borrowed memory address region Y (275) in memory mapping (207) and is not associated with any physical address region in memory mapping (207). Therefore, in order to access virtual address region C (265), the lessee device needs to communicate with the lessor device.

[0090] The contents of physical address region S (283) may be evicted to the lessor device to free up physical address region S (283). Once physical address region S (283) is freed up, it may be used for physical placement of another borrowed memory address region, such as borrowed memory address region Y (275). Once the contents of borrowed memory address region Y (275) are migrated from the lessor device to the lessee device and arrive at physical address region S (283), physical address region S (283) may be used to provide access to, for example, Figure 4 Accesses of virtual address region C (265) are shown mapped to borrowed memory address region Y (275) of the unified address space and physically located in physical address region S (283) of the tenant device.

[0091] exist Figure 5 In the case of a memory management unit (MMU) (e.g., similar to Figure 3 216) has a translation lookaside buffer (TLB) (e.g., similar to Figure 3 251 shown in FIG. 251). When the processor (215) of the tenant device (201) executes an instruction using the virtual address (255), the TLB (251) uses the virtual memory to physical memory mapping (253) to translate the virtual address (255) into a physical address (257); and the memory management unit (MMU) (216) can use the physical address (257) (e.g., via a memory bus) to access a memory page (259) in the memory (211).

[0092] The virtual memory to physical memory mapping (253) loaded in the TLB (251) is typically part of a memory mapping (207) managed by an operating system (213) of the tenant device (201). When a processor (215) of the tenant device (201) executes an instruction that uses a virtual address that is not covered in the virtual memory to physical memory mapping (253) in the TLB (251), a memory management unit (MMU) (216) can cause the processor (215) to execute an operating system (213) that is programmed to replace a portion of the virtual memory to physical memory mapping (253) in the TLB (251) so that the updated virtual memory to physical memory mapping (253) in the TLB (251) contains data for translating a virtual address (e.g., 255) into a physical address (e.g., 257).

[0093] The memory management unit (MMU) (216) is generally configured to access the memory (211) via a memory bus coupled between the memory management unit (216) of the lessee device (201) and the local memory (211) of the lessee device (201). When the virtual address (255) is initially located in a virtual address region (e.g., 265) mapped to a borrowed memory address region (e.g., 275), the operating system (213) can migrate the contents of this region from the borrowed memory of the lessor device to a physical address region (e.g., 283) in the memory (211), and update the virtual memory to physical memory mapping (253) in the TLB (251) to allow the virtual address (255) to be directly translated into a physical address (257) corresponding to the virtual address (255) without translating the virtual address (255) into the borrowed memory address region (e.g., 275).

[0094] In some embodiments, the memory management unit (MMU) (216) is further configured to access the borrowed memory (204) using the communication device (217) through the computer network connection (e.g., 205) without explicit migration. In this case, it is not necessary for the operating system (213) to migrate the contents of the virtual address area (e.g., 265). The physical address (257) of the borrowed memory (204) may contain information for the communication device (217) to access the memory page (260) in the borrowed memory (202) in the lessor device (203). The memory access request made using the physical address (257) and through the communication device (217) may be processed in the lessor device (203) by the operating system (233) of the lessor device. Alternatively, the communication device (237) and / or the memory management unit (MMS) (236) may be configured to process such access requests to the loaned memory (202) (e.g., for reading or writing) without executing the operating system (233) (e.g., based on a virtual memory to physical memory mapping cached in a TLB of the MMS (236) of the lessor device (203) or a hardware acceleration component configured in the lessor device (203) for performing similar operations).

[0095] The mapping (253) of virtual memory to physical memory may be implemented in the form of page table entries.

[0096] Figure 6 A tenant-lessor memory hierarchy in a computing system is presented.

[0097] exist Figure 6 In the example, the lessor device (203) may be configured similar to Figure 2 The lessee device (201) borrows memory from one or more lessor servers (245) in the same manner as borrowing memory from a lessor device (203). The lessor device (203) at least transfers the local physical memory (e.g., Figure 2 Some local physical memory in 231) shown in FIG. 2 and / or some memory borrowed by the lessor device from a lessor server (e.g., 245) are collected as available memory (239) to be lent to the lessee device (201). One or more lessee devices (201) can borrow memory from the lessor device (203).

[0098] A typical borrowed memory (204) may have multiple borrowed memory areas (295, 297, ..., 299) and multiple local memory areas (291, 293, ...). The borrowed memory areas (295, 297, ..., 299) in the lessee device (201) may be accessed by migrating from the borrowed memory of the lessor device to the local memory areas (291, 293, ...) of the lessee device. When the borrowed memory areas (295, 297, ..., 299) are not used in the lessee device (201), the borrowed memory areas may be evicted back to the lessor device (203), such as Figure 4 and Figure 5 As shown in .

[0099] Similarly, the lessor device (203) can host the area (eg, 299) loaned to the lessee device (201) in the local storage of the lessee device or in the local storage borrowed from the lessor server (245).

[0100] Typically, more than one lessor device (203) may be used to provide storage services to a lessee device (201); and more than one lessor server (245) may be configured in a hierarchical structure.

[0101] Optionally, the storage provided as a service in the hierarchy (e.g., 295, 297, ..., 299) is configured in a unified virtual address space. Therefore, the tenant device (201), the lessor device (203), and the lessor server (245) can collaboratively process data in the storage by referring to the virtual addresses in the unified virtual address space.

[0102] For example, Figure 6 The tenant device (201) in may be Figure 1 Device A (101) or device B (103) in the system.

[0103] For example, Figure 6 The lessor device (203) in may be Figure 1 Device B (103) or server P (105) in the system.

[0104] For example, Figure 6 The lessor server (245) in may be Figure 1 Server P (105) or server Q (107) in the system.

[0105] Figure 7 A method for implementing inter-operating system memory services via a communication network connection is shown. For example, Figure 7 The method can be used Figure 2The tenant-lessor configuration shown in Figure 6 The tenant-lessor storage hierarchy shown in Figure 1 implemented in the system.

[0106] At block 301 , a communication connection ( 205 ) is established from a tenant device ( 201 ) to a tenant device ( 203 ).

[0107] At block 303, the lessee device (201) communicates with the lessor device (203) to borrow a certain amount of memory from the lessor device (203). The lessor device (203) may allocate the loan memory (203) borrowed by the lessee device (201) as a borrowing memory (204) that may be used as virtual memory by the borrowing memory (204). By borrowing / lending memory, the devices (201 and 203) establish a configuration in which a processor of the lessee device (201) may read and / or write to the borrowing memory (204) / lending memory (202). In some instances, the loaning memory (202) itself is memory borrowed from another lessor server (e.g., 245).

[0108] At block 305, the operating system (213) of the tenant device (201) expands the virtual / logical address space of the tenant device (201) to cover the borrowed memory (204). Applications (e.g., 212) running on the processor (215) of the tenant device (201) can use virtual addresses in the space without distinguishing between the borrowed memory (204) and the local memory (211).

[0109] At block 307 , the operating system ( 213 ) of the tenant device ( 201 ) generates a memory map ( 207 ) (eg, in the form of a page table) that maps the extended portion of the logical address space to the borrowed memory ( 204 ).

[0110] At block 309 , the operating system ( 213 ) of the tenant device ( 201 ) allocates an extended portion of the logical address space to the application ( 212 ).

[0111] At block 311 , the operating system ( 213 ) of the tenant device ( 201 ) services access to the borrowed memory ( 204 ) by accessing a portion of the physical memory ( 211 ) of the tenant device ( 201 ).

[0112] Figure 8 A method for servicing pages of borrowed memory according to one embodiment is shown. For example, Figure 8 The method can be used Figure 2The tenant-lessor configuration shown in Figure 6 The tenant-lessor storage hierarchy shown in Figure 1 For example, Figure 8 The method can be used to implement Figure 7 Block 311 of the method.

[0113] At block 331 , the tenant device ( 201 ) reserves a page of physical memory ( 211 ) of the tenant device ( 201 ) for the borrowed memory page ( 204 ).

[0114] At block 333, the processor of the lessee device (201) accesses a virtual memory address (255) corresponding to a memory page (260) in the loan memory (202). The borrowed memory page (204) may be physically located in the loan memory (202) of the lessor device (203).

[0115] At block 335, a memory management unit (MMU) (216) of the tenant device (201) determines that there is no available page table entry in the translation lookaside buffer (TLB) (251) to translate the virtual memory address (255) into a local physical memory address. Such a determination (e.g., a TLB miss) causes the tenant device (201) to execute an operating system (213) of the tenant device to search for a page table entry in the memory map.

[0116] At block 337 , the operating system ( 213 ) running on the tenant device ( 201 ) identifies a page table entry in the memory map that translates the virtual memory address of the page to the borrowed memory address of the page.

[0117] At block 339, the operating system (213) running on the lessee device (201) communicates with the lessor device and migrates the physical contents of the borrowing page having the borrowing memory address to the local memory and places the contents at the available physical address of the local memory. When there is not enough local memory to migrate the borrowing page from the lessor device to the lessee device, it may be necessary to perform borrowing page evictions from the lessee device to the lessor device for different borrowing pages.

[0118] At block 341 , the operating system ( 213 ) running on the tenant device ( 201 ) generates a page table entry that maps the virtual address of the borrowed page (eg, 255 ) to a physical address ( 257 ) in local memory.

[0119] At box 343, the operating system (213) running on the tenant device (201) loads the page table entry into the translation lookaside buffer (TLB) (251), which allows the processor to continue accessing the virtual memory address (255) using the translated physical memory address (257).

[0120] In general, each of the devices or servers discussed above (eg, 101, 103, 105, 107, 201, 203) may be implemented as one or more data processing systems.

[0121] A typical data processing system may include an interconnect (eg, a bus and system core logic) that interconnects a microprocessor and memory. A microprocessor may also have an on-chip cache hierarchy.

[0122] The interconnect interconnects the microprocessor and the memory together and also interconnects the microprocessor and the memory to an input / output (I / O) device through an I / O controller. The I / O device may include a display device and / or peripheral devices such as a mouse, keyboard, modem, network interface, printer, scanner, camera, and other devices known in the art. In one embodiment, when the data processing system is a server system, some of the I / O devices such as a printer, scanner, mouse, and / or keyboard are optional.

[0123] The interconnect may include one or more buses connected to each other through various bridges, controllers and / or adapters. In one embodiment, the I / O controller includes a USB adapter for controlling USB (Universal Serial Bus) peripherals and / or an IEEE-1394 bus adapter for controlling IEEE-1394 peripherals.

[0124] The memory may include one or more of: ROM (read only memory), volatile RAM (random access memory), and non-volatile memory such as a hard drive, flash memory, and the like.

[0125] Volatile RAM is typically implemented as dynamic RAM (DRAM), which continuously requires power to refresh or maintain the data in the memory. Nonvolatile memory is typically a magnetic hard drive, magnetic optical drive, optical drive (e.g., DVD RAM), flash memory, 3D crosspoint, or other types of memory systems that maintain data even after power is removed from the system. Nonvolatile memory can also be random access memory.

[0126] The non-volatile memory may be a local device directly coupled to the rest of the components in the data processing system. Non-volatile memory that is remote from the system may also be used, such as a network storage device coupled to the data processing system through a network interface such as a modem or Ethernet interface.

[0127] In the present disclosure, in order to simplify the description, some functions and operations are described as being performed by software code or caused by software code. However, such expressions are also used to specify that the function is generated by the execution of code / instructions by a processor such as any IP block of a microprocessor or SoC (system on chip).

[0128] Alternatively or in combination, functions and operations as described herein may be implemented using dedicated circuitry, such as using an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA), with or without software instructions. Embodiments may be implemented using hardwired circuitry, without software instructions, or in combination with software instructions. Thus, the described techniques are not limited to any specific combination of hardware circuitry and software, nor to any particular source of instructions executed by a data processing system.

[0129] Although one embodiment may be implemented in fully functional computers and computer systems, the various embodiments may be distributed in various forms as a computer product and may apply regardless of the specific type of machine or computer readable media used to actually effect the distribution.

[0130] At least some aspects disclosed may be embodied at least in part in software. That is, the techniques may be performed in a computer system or other data processing system in response to a processor of the computer system or other data processing system, such as a microprocessor, executing sequences of instructions contained in a memory such as a ROM, volatile RAM, non-volatile memory, cache, or remote storage device.

[0131] The routines executed to implement the embodiments may be implemented as part of an operating system or a specific application, component, program, object, module, or sequence of instructions referred to as a "computer program". A computer program generally includes one or more instructions that are placed at various times in various memories and storage devices in the computer and, when read and executed by one or more processors in the computer, cause the computer to perform the operations necessary to execute the elements involved in various aspects.

[0132] Machine-readable media can be used to store software and data, which, when executed by a data processing system, cause the system to perform various methods. Executable software and data can be stored in various places, including, for example, ROM, volatile RAM, nonvolatile memory and / or cache. Portions of this software and / or data can be stored in any one of these storage devices. Further, data and instructions can be obtained from a centralized server or a peer-to-peer network. Different parts of data and instructions can be obtained from different centralized servers and / or peer-to-peer networks at different times and in different communication sessions or in the same communication session. Data and instructions can be obtained as a whole before executing an application. Alternatively, portions of data and instructions can be obtained dynamically and in a timely manner when execution is needed. Therefore, data and instructions do not need to be on a machine-readable medium as a whole at an instance of a specific time.

[0133] Examples of computer readable media include, but are not limited to, non-volatile, recordable and non-recordable types of media, such as volatile and non-volatile memory devices, read-only memory (ROM), random access memory (RAM), flash memory devices, floppy disks and other removable disks, magnetic disk storage media, optical storage media (e.g., compact disk read-only memory (CD ROM), digital versatile disk (DVD), etc.), etc. A computer readable medium may store instructions.

[0134] The instructions may also be embodied in digital and analog communication links using electrical, optical, acoustic or other forms of propagated signals such as carrier waves, infrared signals, digital signals, etc. However, propagated signals such as carrier waves, infrared signals, digital signals, etc. are not tangible machine-readable media and are not configured to store instructions.

[0135] Generally, a machine-readable medium includes any mechanism that provides (ie, stores and / or transmits) information in a form accessible by a machine (eg, a computer, a network device, a personal digital assistant, a manufacturing tool, any device with a set of one or more processors, etc.).

[0136] In various embodiments, hard-wired circuitry can be used in combination with software instructions to implement the techniques. Therefore, the techniques are neither limited to any specific combination of hardware circuitry and software nor to any particular source for instructions executed by a data processing system.

[0137] The above description and accompanying drawings are illustrative and should not be construed as limiting. Many specific details are described to provide a thorough understanding. However, in some cases, well-known or conventional details are not described to avoid obscuring the description. References to one or an embodiment in the present disclosure are not necessarily references to the same embodiment; and such references mean at least one.

[0138] In the foregoing description, the present disclosure has been described with reference to specific exemplary embodiments of the present disclosure. It will be apparent that various modifications may be made to the present disclosure without departing from the broader spirit and scope as set forth in the following claims. Therefore, the description and drawings should be regarded as illustrative rather than restrictive.

Claims

1. A computing device, wherein the computing device is a lessee device, and the lessee device comprises: a communication device configured to communicate via a wired or wireless network connection; Local storage; as well as at least one microprocessor coupled to the local memory, the microprocessor having a memory management unit configured to convert a virtual memory address used by the microprocessor into a physical memory address of the local memory and to access the local memory according to the physical memory address converted from the virtual memory address; wherein the tenant device is configured to: Using the communication device to communicate with the lessor's device; accessing a certain amount of memory at the lessor device; expanding a memory capacity of the tenant device for an application running on the at least one microprocessor using an amount of local memory on the tenant device and an amount of memory on the lessor device; and Access to the virtual storage by the application is serviced by communications between the tenant device and the lender device through the communications device.

2. The computing device of claim 1, wherein the tenant device is further configured to: allocating a page of virtual memory to an application and mapping the page to a page in loan memory of the lessor device, the loan memory being a portion of memory of the lessor device that is loaned or provided by the lessor device to the tenant device for memory allocation and access; and Pages are migrated from the loan memory of the lessor device to pages in the local memory in the tenant device in response to executing the application in the microprocessor using the pages of the virtual memory mapped to the pages of the loan memory.

3. The computing device of claim 2, wherein the tenant device is further configured to: migrating the page of the virtual memory mapped to the local memory of the tenant device from the local memory of the tenant device to a page of the loaned memory on the lessor device after the page of the virtual memory mapped to the page of the local memory has not been used for a period of time; and The migrated page is remapped to the page of the loaned memory on the lender device.

4. The computing device of claim 1 , wherein the memory management unit includes a translation lookaside buffer, the translation lookaside buffer configured to: cache a portion of a page table; in response to the translation lookaside buffer or the at least one microprocessor identifying a page fault when accessing a page of virtual memory, the tenant device is further configured to: retrieving page contents in a page of a loan memory stored in the lessor device by communicating with the lessor device through the communication device; allocating pages in the local memory of the tenant device; storing the retrieved borrowing page content in the page allocated in the local memory, the borrowing page being a page whose content is retrieved from the lending memory of the lessor device and transmitted to the lessee device through the communication device; generating a page table entry, the page table entry mapping the page of the virtual memory to the page allocated in the local memory; storing the page table entry in the page table in the local memory; and The page table entry is loaded into the translation lookaside buffer to resolve the page fault.

5. The computing device of claim 4, wherein the tenant device is further configured to: The borrowed page is deallocated from the local memory of the tenant device.

6. The computing device of claim 4, wherein the tenant device is further configured to: The borrowed page is returned to the lender device.

7. The computing device of claim 4, wherein the tenant device is further configured to: transferring current contents of the borrowing page allocated in the local memory of the tenant device to a page in the lending memory of the tenant device; removing the page table entry from the translation lookaside buffer and the page table of the tenant device; and The page table entry is updated to map a page of the virtual memory to a page of the loaned memory in the lender device.

8. The computing device of claim 4, wherein the tenant device is further configured to: allocating a new page in the loaned memory of the lessor device; transferring current content stored in the page allocated in the local memory to the new page of the loan memory of the lender device; removing the page table entry from the translation lookaside buffer and a page table of the tenant device; and The page table entry is updated to map the page of the virtual memory to the new page in the loan memory of the lender device.

9. A computing device according to claim 1, wherein the microprocessor and the local memory are configured in the same computer chip; the lessee device is further configured to rent the memory to at least one other lessee device; and the lessor device is further configured to borrow the memory from at least one other lessor device.

10. The computing device of claim 1, wherein a portion of an operating system of the tenant device is virtualized for execution in the lessor device by the operating system of the lessor device.

11. The computing device of claim 1, wherein the lessor device provides the amount of memory as a virtual memory device; and the lessee device is further configured to make the virtual memory device available in the lessee device to expand the memory capacity of the lessee device.

12. A computing device, the computing device being a lessor device, the lessor device comprising: at least one communication device configured to communicate via a wired or wireless network connection; Local storage; as well as at least one microprocessor coupled to the local memory, the microprocessor having a memory management unit configured to convert a virtual memory address used by the microprocessor into a physical memory address of the local memory and to access the local memory according to the physical memory address converted from the virtual memory address; The lessor device is configured to: communicating with the lessee device using the at least one communication device; providing the tenant device with access to an amount of local memory of the lessor device to expand the memory capacity of the tenant device; and Access by the tenant device to the amount of local storage of the tenant device is serviced through communications between the tenant device and the tenant device.

13. The computing device of claim 12, wherein the lessor device is further configured to: communicating with the lessee device using the at least one communication device; leasing a certain amount of storage to the lessee device; expanding a memory capacity of the tenant device for an application running on the at least one microprocessor using at least the memory of the tenant device and the amount of local memory of the lessor device; and A portion of access by the tenant device to the amount of local memory of the lessor device is serviced by the memory borrowed by the tenant device from the lessor device and by communications between the lessor device and the tenant device.

14. The computing device of claim 13, further comprising: Random Access Memory; The lessor device is further configured to: providing a portion of the random access memory as a rental amount to the lessee device; and The portion of the random access memory is used to service access by the tenant device to the amount of memory leased to the tenant device.

15. The computing device of claim 14, wherein the random access memory is coupled to a memory bus.

16. The computing device of claim 14, wherein the random access memory is coupled to a peripheral bus; and the lessor device further comprises: A solid state drive includes the random access memory and a controller connected to the peripheral bus.

17. The computing device of claim 12, wherein the lessor device is further configured to borrow memory from at least one additional lessor device, and the lessee device is further configured to rent memory to at least one additional lessee device.

18. A computer system comprising: a first computing device having a local memory and a microprocessor; a second computing device connected to the first computing device via a first communication link; as well as a third computing device having a local memory and connected to the second computing device via a second communication link; wherein the second computing device is configured to borrow a first amount of memory from the third computing device and rent a second amount of memory to the first computing device, the second amount of memory comprising at least the first amount of memory of the third computing device borrowed by the second computing device from the third computing device; and wherein the second computing device is configured to expand the memory capacity of the second computing device using the first amount of memory borrowed from the third computing device; and Wherein the first computing device is configured to expand a memory capacity of the first computing device using the second amount of memory borrowed from the second computing device.

19. A computer system according to claim 18, wherein the first computing device is a wearable computing device, a mobile device or a computing device of the Internet of Things (IoT); the first communication link is a wireless local area network, a wireless personal area network or a local area network; the second computing device is a mobile computing device, a personal computer or a computing device; the second communication link is a local area network or the Internet; and the third computing device is a server computer or a server farm.

20. The computer system of claim 18, wherein at least one of the first amount of memory and the second amount of memory is byte-addressable in a memory management unit of at least one of the first computing device, the second computing device, and the third computing device; and operating systems of the first computing device, the second computing device, and the third computing device communicate with each other to facilitate access to at least one of the first amount of memory and the second amount of memory by an application running on at least one of the first computing device, the second computing device, and the third computing device.

21. The computer system of claim 18, wherein at least one of the first communication link and the second communication link is established through a fifth generation cellular network.

Images