Remote memory cache replacement

Abstract

The invention relates to remote memory cache replacement. In one embodiment, a network device comprises: a network interface for sharing data over a network; the cache memory is used for storing a software code unit; the acquirer circuit is used for acquiring the software code unit and loading the software code unit into the cache; and a processing unit for checking in the cache memory a given software code unit that results in a cache miss, where, as part of a cache replacement process, the retriever circuit is for receiving the given software code unit from a remote memory disposed in a remote device over a network, and for storing the received software code unit in the cache memory. And providing the given software code unit to the cache memory.

Description

Technical Field

[0001] This disclosure relates to computer systems, and more particularly, but not exclusively, to cache replacement. Background Technology

[0002] Devices that include chips with processors to run firmware (FW) may have enough memory to hold the entire firmware image on the chip. The processor can then load the firmware from the chip, authenticate it, and run it. From a software perspective, this is the simplest solution, but it is also the least cost-effective method because storing the entire firmware image on the chip requires a significant amount of memory.

[0003] To address this challenge, central processing units (CPUs) or other processors have caches where code is loaded from off-chip memory (such as DRAM) as needed. When a processor needs to read from or write to a location in memory, it first checks the corresponding entry in the cache. The cache checks the requested memory location in any cache line that might contain that address. If the processor finds the memory location in the cache, a cache hit occurs. However, if the processor cannot find the corresponding memory location in the cache, a cache miss occurs. If a cache hit occurs, the processor reads or writes the data from the cache line. For a cache miss, the cache allocates a new entry, copies the data from main memory (such as DRAM) to the new entry, and executes the request based on the contents of the new entry in the cache. Summary of the Invention

[0004] According to one embodiment of this disclosure, a network device is provided, comprising: a network interface for sharing data over a network; a cache memory for storing software code units; a acquirer circuit for acquiring the software code units to load into the cache memory; and a processing unit for checking in the cache memory for a given software code unit that causes a cache miss, wherein the acquirer circuit is configured to: receive the given software code unit from a remote memory disposed in a remote device via the network as part of a cache replacement process; and provide the given software code unit to the cache memory.

[0005] Furthermore, according to embodiments of this disclosure, the cache memory is used to invoke the acquirer circuit to retrieve the given software code unit as part of the cache replacement process; the acquirer circuit is used to generate a request for the given software code unit from the remote memory disposed in the remote device; the network interface is used to provide the request to the remote device via the network and to receive the given software code unit from the remote device via the network; the acquirer circuit is used to receive the given software code unit and provide the given software code unit to the cache memory; the cache memory is used to store the given software code unit; and the processor unit is used to receive the given software code unit from the cache memory and execute the given software code unit.

[0006] Furthermore, according to one embodiment of this disclosure, the acquirer circuitry is configured to: generate a remote direct memory access request for the given software code unit; provide the remote direct memory access request to the remote device via the network; and receive a remote direct memory access response including the given software code unit from the remote device via the network.

[0007] Furthermore, according to one embodiment of this disclosure, the device includes a non-volatile memory (NVM) controller configured to: read the software code from a local NVM as part of a boot process; and provide the software code to the processing unit, wherein the processing unit is configured to provide the software code to the remote device for writing into the remote memory as part of the boot process.

[0008] Furthermore, according to one embodiment of this disclosure, the processing unit is configured to write the software code into the remote memory using at least one remote direct memory access write operation as part of the startup process.

[0009] Furthermore, according to one embodiment of this disclosure, the device further includes a non-volatile memory (NVM) controller for retrieving data from a local NVM that stores the software code, wherein the retriever circuitry is configured to retrieve the given software code unit from the remote memory rather than from the local NVM.

[0010] Furthermore, according to one embodiment of this disclosure, the acquirer circuitry is configured to generate a first request for another software code unit from the remote memory located in the remote device; and the acquirer circuitry is configured to generate a second request to the NVM controller to retrieve the other software code unit from the local NVM.

[0011] Furthermore, according to one embodiment of this disclosure, the NVM controller is configured to retrieve the other software code unit from the local NVM and provide the other software code unit to the acquirer circuit; the acquirer circuit is configured to receive the other software code unit and provide the other software code unit to the cache memory; the cache memory is configured to store the other software code unit; and the processor unit is configured to receive the other software code unit from the cache memory and execute the other software code unit.

[0012] Furthermore, according to one embodiment of this disclosure, the acquirer circuitry is configured to provide the first request to the remote device, and, after failing to retrieve the other software code unit from the remote memory in response to the first request, provide the second request to the NVM controller.

[0013] Furthermore, according to one embodiment of this disclosure, the acquirer circuitry is configured to determine the failure of the first request based on at least one configured strategy, the at least one configured strategy including one or more of the following: a maximum number of retries to receive the other software code unit in response to the first request; a timeout period for waiting to receive the other software code unit in response to the first request; identification of network problems; identification of problems with the remote device; and identification of errors in the other software code unit received from the remote memory.

[0014] Furthermore, according to one embodiment of this disclosure, the acquirer circuitry is configured to execute acquirer software code that performs cache replacement only from the local NVM.

[0015] Furthermore, according to one embodiment of this disclosure, the network device is a network switch.

[0016] Furthermore, according to one embodiment of this disclosure, the remote memory is dynamic random access memory (DRAM).

[0017] Furthermore, according to one embodiment of this disclosure, the acquirer circuit is configured to receive packets containing the given software code units out of order.

[0018] Furthermore, according to one embodiment of this disclosure, the given software code unit is an authenticable software code unit, and the acquirer circuitry is configured to authenticate the given software code unit after receiving all packets of a given request from the remote memory.

[0019] Furthermore, according to one embodiment of this disclosure, the acquirer circuitry is configured to receive the given software code unit from the remote memory via an unreliable connection to the network.

[0020] Furthermore, according to one embodiment of this disclosure, the device includes a read-only memory (ROM) containing boot management and recovery routines.

[0021] Furthermore, according to one embodiment of this disclosure, the acquirer circuitry is configured to restrict the reception of the given software code unit to at least one of the following: a given network; or a given port connection of the remote device.

[0022] Furthermore, according to one embodiment of this disclosure, the acquirer circuit is configured to receive the given software code unit in encrypted form from the remote device.

[0023] Furthermore, according to one embodiment of this disclosure, the acquirer circuit is configured to: receive the given software code unit in unencrypted form from the remote device, and authenticate the given software code unit before executing it.

[0024] Furthermore, according to one embodiment of this disclosure, the acquirer circuitry is configured to attempt to retrieve another software code unit from multiple remote memories on multiple corresponding remote devices via the network as part of the cache replacement process.

[0025] According to another embodiment of this disclosure, a method is also provided, the method comprising: acquiring software code units of software code to load into a cache memory; checking in the cache memory for a given software code unit that causes a cache miss; receiving the given software code unit from a remote memory located in a remote device via a network as part of the cache replacement process; and providing the given software code unit to the cache memory. Attached Figure Description

[0026] This disclosure can be understood from the following detailed description, taken in conjunction with the accompanying drawings, wherein:

[0027] Figure 1 This is a block diagram view of a cache replacement system built and operated according to embodiments of this disclosure;

[0028] Figure 2 It includes Figure 1 A data flow diagram of the data flow in the system;

[0029] Figure 3 It includes Figure 1 A flowchart of the steps involved in the startup process of network devices in the system;

[0030] Figure 4 It includes Figure 1 A flowchart of the steps in the cache replacement method in the system;

[0031] Figure 5 It is built and operated according to embodiments of this disclosure. Figure 1 A schematic diagram of the data center in the system. Detailed Implementation

[0032] Overview

[0033] As mentioned earlier, a device running firmware should ideally have enough on-chip memory to store the entire firmware image. However, this may not be achievable, primarily due to cost considerations.

[0034] One solution is to use off-chip memory, such as DRAM, for example, by using off-chip memory on the connected host device. The host device can share memory (e.g., DRAM) with a specific device (e.g., a network interface controller (NIC)). The host device can provide some host memory for the NIC to use, for example, by uploading firmware code from non-volatile memory (NVM) (e.g., a flash drive) to the host memory, and then the NIC retrieves the firmware code units as needed. The reason for doing this instead of directly using flash memory is that accessing DRAM on the host device is much faster than accessing flash memory attached to the NIC. Accessing flash memory can be one or two orders of magnitude slower than accessing DRAM.

[0035] In some cases, such as when using a network switch, a device running firmware may not be able to use local DRAM, so it may be necessary to use NVM to store the firmware image. The device will then load the firmware from NVM, resulting in slow access to the firmware.

[0036] Therefore, embodiments of this disclosure address at least some of the aforementioned deficiencies by providing a network device that can access software code units (e.g., firmware) from remote memory (e.g., DRAM) over a network, for example using Remote Direct Memory Access (RDMA). Accessing software units from remote memory over a network can be much faster than accessing software units from a local NVM, even when connected to the network device via a non-network connection.

[0037] In some embodiments, the network device may be a network switch with a high-speed architecture, with each port capable of hundreds of gigabits. The network device may use its network interface (which is hundreds of times faster than flash memory) to access remote memory (e.g., DRAM) allocated by the CPU in a remote device to fetch missing code segments into its cache at runtime, and to authenticate and execute the received code.

[0038] In some cases, the requested software unit may fail to arrive (e.g., due to network problems or issues with the remote device), may arrive from the remote device with an error, may arrive out of order, or may arrive after a delay. Therefore, in some embodiments, the network device may request the software unit from the local NVM after failing to receive it from the remote memory. Requesting the software unit from the local NVM can be performed as a supplement to or alternative to re-requesting the software unit from the remote device. In some embodiments, after failing to receive the software unit from the original remote memory, the software unit can be requested from another remote memory.

[0039] In some embodiments, a policy is established to define what constitutes a "failure" in receiving a software unit and how to proceed when a failure is detected. A failure can be defined relative to a given number of attempts and / or timeout. A failure may result in requesting the software unit from the local NVM, and / or re-requesting the software unit from the original remote storage, and / or requesting the software unit from another remote storage. In some embodiments, if a network error is detected, or an error is detected in a remote device or remote storage, the network device may default to requesting the software unit from the local NVM until the error is resolved.

[0040] In some embodiments, during network device startup, software code stored in the local NVM is copied by the network device to remote memory (e.g., using RDMA). The startup process may also include configuring the network device to take action according to a determined "failure" policy. At runtime, if a processing unit of the network device needs a given software code unit, the processing unit verifies whether the given software code unit is located in the network device's cache memory. If the given software code unit is located in the cache memory, the processing unit reads the given software code unit from the cache memory. If the given software code unit is not located in the cache memory, a cache miss occurs, and a fetcher circuit is invoked to retrieve the given software code unit. The fetcher logic initiates a process to retrieve the given software code unit from remote memory, for example, using an RDMA request. If retrieving a given software code unit fails, for example, due to some event (e.g., a remote system failure, network problems, or latency) causing the remote memory on the remote device to become unavailable, the acquirer circuitry will (explicitly or implicitly) recognize the failure and, instead of waiting for the remote memory to become available, the acquirer circuitry will retrieve the given software code unit from the network device's local non-volatile memory via the local non-volatile memory controller. The acquirer logic will then load the retrieved given software code unit into the cache memory.

[0041] System Description

[0042] Now for reference Figure 1 , Figure 1 This is a block diagram view of a cache replacement system 10 constructed and operated according to embodiments of the present disclosure. System 10 includes a network device 12 and a remote device 14. In some embodiments, system 10 includes one or more other remote devices 16.

[0043] Network device 12 includes a processing unit 18, a cache memory 20, a read-only memory (ROM) 22, a network interface 24, a pickup circuit 26, and a non-volatile memory (NVM) controller 28. Network device 12 also includes an integrated circuit chip 30, such as an application-specific integrated circuit (ASIC) chip, like a network switch ASIC. Network device 12 can be any suitable network device, such as a network switch or a network interface controller (NIC).

[0044] Processing unit 18 is configured to execute software code, such as firmware code. Processing unit 18 can be any suitable processing unit, such as a central processing unit (CPU). (See reference...) Figures 2 to 4 The processing unit 18 is described in more detail.

[0045] In practice, some or all of the functions of processing unit 18 may be combined in a single physical component, or alternatively implemented using multiple physical components. These physical components may include hardwired devices or programmable devices, or a combination of both. In some embodiments, at least some of the functions of processing unit 18 may be executed by a programmable processor under the control of suitable software. The software may be downloaded to the device electronically, for example, via a network. Alternatively or additionally, the software may be stored in a tangible, non-transitory computer-readable storage medium, such as optical, magnetic, or electronic memory.

[0046] The cache memory 20 is configured to store software code units (SWCUs) of software code according to a caching strategy that will refer to... Figure 2 and Figure 4 A more detailed description follows. Cache memory 20 contains cache lines (not shown) representing arbitrary locations in the software code space.

[0047] ROM 22 is a non-volatile memory hard-coded on network device 12 and contains boot management and recovery routines, for example, to support recovery of local NVM 32 in the event of damage to local NVM 32 (e.g., flash memory). Network interface 24 is configured to share data with remote device 14 and one or more remote devices 16 via network 34.

[0048] The acquirer circuit 26 is configured to fetch the SWCU and load it into the cache memory 20. In some embodiments, the acquirer circuit 26 is configured to execute acquirer software code (not shown) to provide at least a portion of the functionality of the acquirer circuit 26. The acquirer software code typically performs cache replacement from the local NVM 32 rather than remote memory, because cache replacement from the remote device 14 could result in deadlock. The acquirer software code may be configured to implement remote direct memory access (RDMA) functionality, such as generating RDMA read and write operations and processing RDMA responses. In other embodiments, the acquirer circuit 26 may be implemented using firmware running on a suitable processor, or the acquirer circuit 26 may be implemented in hardware.

[0049] In practice, some or all of the functionality of the acquirer circuitry 26 may be combined in a single physical component, or alternatively, it may be implemented using multiple physical components. These physical components may include hardwired devices or programmable devices, or a combination of both. In some embodiments, at least some of the functionality of the acquirer circuitry 26 may be executed by a programmable processor under suitable software control. This software may be downloaded to the device electronically, for example, via a network. Alternatively or additionally, the software may be stored in a tangible, non-transitory computer-readable storage medium, such as optical, magnetic, or electronic memory.

[0050] NVM controller 28 is configured to retrieve data (e.g., software code) from local NVM 32, which is configured to store software code.

[0051] Remote device 14 includes a network interface 36 and remote memory 38. Similarly, each remote device 16 includes a network interface 40 and remote memory 42. Each remote device 14, 16 typically includes a processor, such as a CPU and / or a graphics processing unit (GPU). Remote device 14 may be a network switch or host device, may include a network switch or host device, or may be included in a network switch or host device. Remote memory 38, 42 may be any suitable memory, such as dynamic random access memory (DRAM). Network interfaces 36, 40 are configured to share data with network device 12 via network 34.

[0052] In some embodiments, multiple copies of the software code image may be remotely stored on network device 12 via a network, for example, in remote memory 38 and / or remote memory 42. The software code image is typically copied from the local NVM 32 to remote memory 38 and / or remote memory 42 by processing unit 18 as part of the boot process, which will refer to... Figure 2 and Figure 3A more detailed description follows. Multiple copies of the software code are remotely stored on network device 12, enabling network device 12 to retrieve the SWCU from different remote memories in the event of a failure of one of the remote memories (or a connection to one of the remote memories).

[0053] Now for reference Figure 2 , Figure 2 It includes Figure 1 Data flow diagram 200 for the data flow in System 10. Also refer to... Figure 1 Network device 12 loads boot code (arrow 202) from the local NVM 32 and runs it (arrow 204). Network device 12 loads software code from the local NVM 32 and writes it to remote memory 38 (arrow 206). Network device 12 configures a remote access policy that specifies when the acquirer circuit 26 should retrieve the SWCU from the local NVM 32 by default instead of from the remote memory 38 (arrow 208). The remote access policy will refer to... Figure 4 A more detailed description follows. Depending on the remote access policy, the acquirer circuit 26 selectively retrieves the SWCU from the remote memory 38 (arrow 210) and the local NVM 32 (arrow 212).

[0054] Now for reference Figure 3 , Figure 3 It is flowchart 300, including Figure 1 The steps of the startup process for network device 12 in system 10 are shown. NVM controller 28 is configured to: read software code from local NVM 32 as part of the startup process (block 302); and provide the software code to processing unit 18 (block 304). Processing unit 18 is configured to provide the software code to remote device 14 for writing to remote memory 38 as part of the startup process (block 306). In some embodiments, processing unit 18 is configured to write the software code to remote memory 38 using at least one RDMA write as part of the startup process (block 308).

[0055] Now for reference Figure 4 , Figure 4 It is flowchart 400, which includes Figure 1The steps of the cache replacement method in System 10 are shown below. Processing unit 18 is configured to check a given SWCU in cache memory 20 (block 402). The given SWCU may be one or more computer instructions and / or verifiable code units. At decision block 404, it is determined whether the check of the given SWCU in cache memory 20 results in a cache miss or a cache hit. If a cache hit occurs, processing unit 18 receives the given SWCU from cache memory 20 and executes the SWCU (block 406). If a cache miss occurs, cache memory 20 is configured to invoke acquirer circuit 26 to retrieve the given SWCU as part of the cache replacement process (block 408). At decision block 409, acquirer circuit 26 checks whether remote memory 38 has been initialized (i.e., whether all software code has been loaded onto remote memory 38). If all software code has not been loaded onto remote memory 38, the process continues with the steps in block 426, loading the SWCU from the local NVM 32. If all software has been loaded onto remote memory 38, acquirer circuitry 26 is configured to generate a request to receive a given SWCU from remote memory 38 located in remote device 14 (block 410). In some embodiments, acquirer circuitry 26 is configured to generate a Remote Direct Memory Access (RDMA) request (e.g., an RDMA read request) to receive the given SWCU. In some embodiments, acquirer circuitry 26 is configured to limit the receipt of the given SWCU from remote memory 38 to at least one of: a given network; or a given port connection of remote device 14. Acquirer circuitry 26 is configured to provide the remote direct memory access request to remote device 14 via network interface 24 through network 34 (block 412), the network interface 24 being configured to provide the request to remote device 14 via network 34.

[0056] In some embodiments, any suitable direct or even indirect access method can be used instead of RDMA to retrieve a given SWCU. For example, a given SWCU can be received via a TCP connection or even via an unreliable transport protocol (e.g., UDP with special priority), where the acquirer circuit 26 requests the CPU of the remote device 14 to provide the SWCU. If the SWCU does not arrive or arrives with an error, the SWCU can be re-requested and / or retrieved from the local NVM 32, as described in more detail below.

[0057] In block 414, the acquirer circuit 26 awaits a response from the remote memory 38. Subsequent steps in blocks 416-422 assume that a given SWCU is received error-free from the remote memory 38. Network interface 24 is configured to receive the given SWCU from the remote memory 38 of the remote device 14 via network 34 as part of a cache replacement process (block 416), and provide the given SWCU to the acquirer circuit 26. In some embodiments, the acquirer circuit 26 is configured to receive a Remote Direct Memory Access (RDMA) response containing the given SWCU from the remote device 14 via network interface 24 through network 34.

[0058] In some embodiments, the acquirer circuit 26 is configured to authenticate a given SWCU (Authenticable Software Code Unit) before executing it (block 417). The acquirer circuit 26 is configured to authenticate a given SWCU after receiving all packets of a given software unit from the remote memory 38. If authentication fails, the SWCU can be requested from a different source (e.g., local NVM 32 and / or a different remote memory 42).

[0059] Network 34 can be a private network or a public network. Network 34 can have one or more layers of encapsulation, and system 10 can use any suitable network functions, such as redirection, virtualization, logical networking, isolation, any structure type (InfiniBand, Ethernet), etc.

[0060] In some cases, the acquirer circuit 26 may receive packets containing a given SWCU in either order or out of order. The acquirer circuit 26 is configured to receive a given SWCU from a remote memory 38 via a reliable or unreliable connection to network 34. In this manner, the acquirer circuit 26 is configured to retrieve a given SWCU from remote memory 38 rather than from the local NVM 32. It should be noted that although the received SWCUs may be received out of order, authentication of the received SWCUs can only be performed after all packets from the authenticable unit have been received.

[0061] The acquirer circuit 26 is configured to provide a given SWCU to the cache memory 20 (block 418). In some embodiments, the acquirer circuit 26 is configured to receive a given SWCU in encrypted form from the remote device 14. In other embodiments, the acquirer circuit 26 is configured to receive a given SWCU in unencrypted form from the remote device 14.

[0062] The cache memory 20 is configured to store a given SWCU (block 420). The processing unit 18 is configured to receive the given SWCU from the cache memory 20 and execute the given SWCU (block 422).

[0063] As previously described, at block 414, the acquirer circuit 26 awaits a response from the remote device 14. If the remote device 14 fails to provide the SWCU within a given timeout period and / or a given number of retries, or if the received SWCU is erroneous, or a network error or other error is detected, the acquirer circuit 26 is configured to determine the failure of the request generated in the step of block 410 (block 424). Upon determining the failure, the acquirer circuit 26 may re-request the SWCU from the remote device 14, and / or request the SWCU from the local NVM 32, and / or request the SWCU from one or more remote memories 42 of one or more other remote devices 16, as described in more detail below.

[0064] In some embodiments, the acquirer circuit 26 is configured to determine the failure of a request (the request generated in the step of block 410) based on at least one configured policy (e.g., a remote access policy). The configured policy may include one or more of the following: a maximum number of retries to receive a SWCU in response to a request; a timeout period for waiting to receive a SWCU in response to a request; identification of network problems; identification of problems with remote device 14; and identification of errors in SWCUs received from remote memory 38.

[0065] The steps described with reference to blocks 426-436 describe retrieving the SWCU from the local NVM 32. The acquirer circuit 26 is configured to generate a second request to the NVM controller 28 to retrieve the SWCU from the local NVM 32 (block 426), and provide the second request to the NVM controller 28. The NVM controller 28 is configured to receive the second request and retrieve the SWCU from the local NVM 32 (block 428), and provide the SWCU to the acquirer circuit 26. The acquirer circuit 26 is configured to receive the SWCU and optionally authenticate the SWCU (block 430). If authentication fails, the network device 12 may enter an error recovery state from which it cannot automatically recover. The acquirer circuit 26 is configured to provide the (authenticated) SWCU to the cache memory 20 (block 432). The cache memory 20 is configured to receive the SWCU and store it in the cache memory 20 (block 434). The processing unit 18 is configured to receive the SWCU from the cache memory 20 and execute the SWCU (block 436).

[0066] In the scenario described above, the acquirer circuit 26 is configured to provide an initial request (the initial request generated in the step of block 410) to the remote device 14, and then, after failing to retrieve the SWCU from the remote memory 14 in response to the initial request for the SWCU, provide a second request to the NVM controller 28 to retrieve the SWCU from the local NVM 32.

[0067] In some embodiments, after a failure to retrieve the SWCU from remote memory 38 in response to an initial request for the SWCU, the acquirer circuitry 26 is configured to provide a third request to remote device 14 (and / or remote device 16) to retrieve the SWCU from remote memory 38 (and / or one or more remote memories 42), and a second request to NVM controller 28 to retrieve the SWCU from local NVM 32. The second and third requests can be provided to NVM controller 28 and remote device 14 (and / or one or more remote devices 16), respectively, without waiting for a response to either the third or second request.

[0068] In some embodiments, the acquirer circuitry 26 is configured to attempt to retrieve one or more SWCUs from multiple remote memories 42 in multiple corresponding remote devices 16 via network 34 as part of a cache replacement process. The attempt to retrieve one or more SWCUs from the multiple remote memories 42 may be performed as part of an initial request, or as a subsequent request sent after an initial request to remote device 14 fails. The request sent to remote device 16 may be sent in lieu of a request sent to NVM controller 28, or as a supplement to a request sent to NVM controller 28.

[0069] Some or all of the functions performed by the remote device 14 may be performed by one or more graphics processing units (GPUs).

[0070] Now for reference Figure 5 , Figure 5 It is built and operated according to embodiments of this disclosure. Figure 1 A schematic diagram of data center 600 in system 10. Figure 1 Devices 12 and 14 can be placed in any suitable environment, such as data center 600. Data center 600 may include rack 602, which may include, for example, devices 12 and 14 (e.g., network switch 604) and terminal host devices 606. Data center 600 may also include cooling systems, power supplies, network components (e.g., the NIC (of terminal host device 606)) and cabling 608 (partially labeled) to provide high-speed connectivity (e.g., connectivity with multiple Internet providers for redundancy), physical and network protection (including access control and monitoring), and organizational space for servers and devices. The data center can support remote storage and computing for cloud services.

[0071] A NIC can include any of the following: an Ethernet port (e.g., using an RJ45 connector), which is the physical interface through which a network cable (typically an Ethernet cable) connects to the NIC for wired network connections; packet processing hardware or circuitry responsible for handling network communications and processing incoming and outgoing data packets and managing network interface functions; memory (e.g., RAM or ROM) for storing temporary data (e.g., network packet buffers, configuration settings, and firmware) and helping to speed up data transfer and processing; firmware, which is software programmed into the NIC's memory to control hardware operation and perform firmware updates to improve performance or add new features to the NIC; and LED indicators. The network provides visual indicators of network status, common indicators include power status, network activity, and link speed; bus interfaces (such as PCI or PCIe) for connecting the NIC to the host motherboard; a processor for handling network processing tasks and other processing tasks to offload the host device's main CPU and improve network performance; a heatsink or cooling mechanism (e.g., for high-performance NICs), especially those used in servers, to prevent overheating; power management circuitry to ensure the NIC receives the correct amount of power and manages power consumption efficiently; and / or connector pins and circuitry, including internal connections and paths for routing signals between the NIC's components.

[0072] Packet processing hardware or circuitry is a core component of the NIC and handles network communications. It may include several key components that work together to manage and process network data, such as any one or more of the following: a MAC (Media Access Control) layer, responsible for handling the data link layer of the OSI model and managing the formatting, addressing, and transmission of data packets over the network; a MAC address register, used to store the NIC's unique hardware address (MAC address); a frame buffer, used to temporarily store data frames during processing; a PHY (Physical Layer) interface, used to interface with the physical medium (e.g., Ethernet cable) and responsible for the actual transmission and reception of data bits over the network; a transceiver, used to convert data between digital signals used at the MAC layer and analog signals used for transmission over the network medium; a DMA (Direct Memory Access) controller, used to manage data transfer between the NIC and computer memory without CPU involvement, helping to offload CPU processing tasks and improve data transfer efficiency; and a packet processing engine, which handles the encapsulation and decapsulation of network packets, processes incoming and outgoing packets, and manages error handling. Tasks include false positive checking and packet filtering; buffer management, including memory areas for temporary packet storage, such as a transmit buffer for storing packets being sent from the computer to the network, and a receive buffer for storing packets received from the network (before the system processes the packets); an interrupt controller responsible for managing and generating interrupts to notify the CPU of events such as packet reception or transmission completion, and facilitating efficient handling of network events; a clock generator that provides timing signals to the various components of the NIC to synchronize their operation; a power management unit that regulates power consumption and manages the power-saving functions of the NIC chip to improve energy efficiency; error handling and correction logic that detects and corrects errors in data transmission and reception, and may include functions for error checking protocols (such as CRC (Cyclic Redundancy Check)); configuration registers that store configuration settings and parameters controlling NIC operation, such as speed settings, interrupt configuration, and buffer size; and firmware / ROM, which contains embedded software that controls NIC operation and manages network protocols.

[0073] A network switch may contain any of the following components: ports for connecting network cables; a switching structure for managing data transfer between ports; a MAC address table for storing device addresses and port information; a forwarding engine for directing data packets to the correct port; a buffer for temporarily storing data to manage traffic; a management processor for handling configuration and monitoring within the managed switch; a power supply; a cooling system to prevent the switch from overheating; firmware to control the switch; LED indicators to display status and activity; and networking modules (located in modular switches) that allow for additional ports or functions.

[0074] Regarding the Graphics Processing Unit (GPU), it is used to generate three-dimensional (3D) and two-dimensional (2D) graphical objects for a variety of applications, including film, computer games, virtual reality (VR) and augmented reality (AR) experiences, mechanical design, and more. Modern GPUs include texture processing hardware for generating surface appearances (referred to as "surface textures" in this paper) for 3D objects in 3D graphics scenes. The texture processing hardware applies a surface appearance to 3D objects by "wrapping" an appropriate surface texture around them. This process of generating surface textures and applying them to 3D objects gives 3D objects in 3D graphics scenes a highly realistic appearance.

[0075] Texture processing hardware is configured to execute various texture-related instructions, including texture manipulation and texture loading. The texture processing hardware accesses texture information by generating a memory reference to the texture memory (referred to herein as a "query"). The texture processing hardware retrieves surface texture information from the texture memory in various situations, such as when rendering object surfaces in a 3D graphics scene for display on a display device, when rendering a 2D graphics scene, or during computational operations.

[0076] Surface texture information includes texture elements (referred to as "texels" in this paper) used for texturing or shading the surfaces of objects in a 3D graphics scene. The texture processing hardware and its associated texture cache are optimized for efficient, high-throughput read-only access to support the large demands on texture information during graphics rendering, while providing little or no support for write operations. Furthermore, the texture processing hardware includes dedicated functional units to perform various texture operations, such as level of detail (LOD) calculations, texture sampling, and texture filtering.

[0077] Generally, texture operations involve querying multiple texels around a specific point of interest in 3D space, then performing various filtering and interpolation operations to determine the final color at that point of interest. In contrast, texture loading typically queries a single texel and returns it directly to the user application for further processing. Because filtering and interpolation operations often involve querying four or more texels per processing thread, texture processing hardware is typically built to accommodate multiple queries per thread. For example, texture processing hardware might be built to accommodate a maximum of four texture memory queries within a single memory cycle. In this way, texture processing hardware is able to query and receive most or all of the required texture information within a single memory cycle.

[0078] In practice, some or all of these functions may be combined in a single physical component, or alternatively implemented using multiple physical components. These physical components may include hardwired or programmable devices, or a combination of both. In some embodiments, at least a portion of the functions of the processing circuitry may be executed by a programmable processor under suitable software control. This software may be downloaded to the device electronically, for example, via a network. Alternatively or additionally, the software may be stored in a tangible, non-transitory computer-readable storage medium, such as optical, magnetic, or electronic memory.

[0079] Implementation of the methods and / or systems of this disclosure may involve performing or completing selected tasks manually, automatically, or in a combination of both. Furthermore, the actual instruments and equipment of the methods and / or systems of this disclosure may be used with an operating system or a cloud-based platform, via hardware, software, firmware, or a combination of both, to perform several selected tasks.

[0080] For example, the hardware for performing selected tasks of the examples of this disclosure can be implemented as a chip or circuit. As software, the selected tasks of the examples of this disclosure can be implemented as multiple software instructions executed by a computer using any suitable operating system. In exemplary examples of this disclosure, one or more tasks according to exemplary examples of the methods and / or systems described herein are performed by a data processor (e.g., a computing platform for executing multiple instructions). Optionally, the data processor includes volatile memory for storing instructions and / or data and / or non-volatile memory for storing instructions and / or data, such as non-transitory storage media, such as magnetic hard disks and / or removable media. Optionally, network connectivity is also provided. Optionally, a display and / or user input device, such as a keyboard or mouse, are also provided.

[0081] For example, according to the examples above in this disclosure, any combination of one or more non-transitory computer-readable (storage) media can be used. A non-transitory computer-readable (storage) medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include the following: an electrical connection having one or more lines, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium can be any tangible medium that can contain or store a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0082] Computer-readable signal media can include propagated data signals in which computer-readable program code is embedded, for example, in baseband or as part of a carrier wave. Such propagated signals can take many forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. Computer-readable signal media can be any computer-readable medium other than a non-computer-readable storage medium, which can communicate, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0083] As can be understood in conjunction with the paragraphs provided above and the accompanying drawings, this document provides examples of various computer implementations of methods, some of which can be executed by the various apparatus and system examples described herein, while others can be executed according to instructions stored in the non-transitory computer-readable storage medium described herein. However, some examples of computer implementations provided herein can be executed by other apparatus or systems and can be executed according to instructions stored in computer-readable storage media other than those described herein, as will become apparent to those skilled in the art upon reference to the examples described herein. Any references to systems and computer-readable storage media relating to the following computer implementations are for illustrative purposes and are not intended to limit the application of any such systems and any such non-transitory computer-readable storage media in the examples of the computer implementations described above. Similarly, any references to the following computer implementations of systems and computer-readable storage media are for illustrative purposes and are not intended to limit any such computer implementations disclosed herein.

[0084] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various examples of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, code segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions shown in the blocks may differ from the order shown in the figures. For example, two blocks shown consecutively may actually execute substantially simultaneously, or sometimes may execute in reverse order. The block diagrams and / or flowchart illustrations, and combinations of blocks in block diagrams and / or flowcharts, can be implemented by a dedicated hardware system or a combination of dedicated hardware and computer instructions that performs the specified functions or actions. The descriptions of the various examples in this disclosure are for illustrative purposes only and are not intended to be exhaustive or limited to the disclosed examples. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the examples.

[0085] The singular forms “a,” “one,” and “the” used in this article include plural references unless the context clearly indicates otherwise.

[0086] It should be understood that certain features of this disclosure, described in separate example contexts for clarity, may also be provided in combination in a single example. Conversely, various features of this disclosure, described in separate example contexts for brevity, may also be provided individually, or in any suitable sub-combination, or appropriately provided in any other example described in this disclosure. Certain features described in various example contexts should not be considered essential features of those examples unless the example cannot be implemented without these elements.

[0087] The processes described above (including parts thereof) can be executed by software, hardware, or a combination thereof. These processes and parts thereof can be executed by computers, computer-like devices, workstations, cloud-based platforms, processors, microprocessors, other electronic search tools, and associated memory and other non-transitory storage devices. These processes and parts thereof can also be embodied in programmable non-transitory storage media, such as optical discs (CDs) or other machine-readable disks, or other computer-usable storage media, including magnetic, optical, or semiconductor storage, or other electronic signal sources.

[0088] This document describes processes (methods) and systems (including their components) by way of example with reference to specific hardware and software. These processes (methods) have been described as exemplary so that those skilled in the art can omit and / or modify specific steps and their order to simplify these examples to practice without requiring excessive experimentation. The processes (methods) and systems have been described in a manner sufficient to allow those skilled in the art to easily adapt them to other hardware and software, thereby reducing the practice of any example as needed without requiring excessive experimentation and the use of conventional techniques.

[0089] For clarity, the various features of this disclosure are described in individual embodiments, or may be provided in combination in one embodiment. Conversely, for brevity, the various features of this disclosure are described in the context of a single embodiment, or may be provided individually or in any suitable sub-combination.

[0090] The above embodiments are merely illustrative examples, and the scope of this disclosure is not limited to the specific content shown and described above. Rather, the scope of this disclosure includes combinations and sub-combinations of the features described above, as well as variations and modifications that can be conceived by those skilled in the art after reading the foregoing description and that are not disclosed in the prior art.

Claims

1. A network device, comprising: A network interface, used to share data over a network; Cache memory, which is a unit of software code used to store software code; A acquirer circuit is used to acquire the software code unit for loading into the cache memory; as well as A processing unit configured to examine a given software code unit in the cache memory that causes a cache miss, wherein the acquirer circuitry is configured to: The given software code unit is received from a remote memory located in a remote device via the network as part of a cache replacement process; and The given software code unit is provided to the cache memory.

2. The device according to claim 1, wherein: The cache memory is used to invoke the acquirer circuit to retrieve the given software code unit as part of the cache replacement process; The acquirer circuit is used to generate a request for the given software code unit from the remote memory located in the remote device; The network interface is used to provide the request to the remote device through the network, and to receive the given software code unit from the remote device through the network; The acquirer circuit is used to receive the given software code unit and provide the given software code unit to the cache memory; The cache memory is used to store the given software code unit; as well as The processor unit is configured to receive the given software code unit from the cache memory and execute the given software code unit.

3. The device according to claim 1, wherein, The acquirer circuit is used for: Generate a remote direct memory access request for the given software code unit; The remote direct memory access request is provided to the remote device via the network; and Receive a remote direct memory access response, including the given software code unit, from the remote device via the network.

4. The device of claim 1, further comprising a non-volatile memory (NVM) controller, which is used for: The software code is read from the local NVM as part of the startup process; and The software code is provided to the processing unit, wherein... The processing unit is used to provide the software code to the remote device so that it can be written into the remote memory as part of the boot process.

5. The device according to claim 4, wherein, The processing unit is used to write the software code into the remote memory using at least one remote direct memory access write operation as part of the boot process.

6. The device of claim 1, further comprising a non-volatile memory (NVM) controller for retrieving data from a local NVM storing the software code, wherein the retriever circuitry is configured to retrieve the given software code unit from the remote memory rather than from the local NVM.

7. The device according to claim 6, wherein: The acquirer circuit is used to generate a first request for another software code unit from the remote memory located in the remote device; and The acquirer circuitry is configured to generate a second request to the NVM controller to retrieve the other software code unit from the local NVM.

8. The device according to claim 7, wherein: The NVM controller is used to retrieve the other software code unit from the local NVM and provide the other software code unit to the acquirer circuit; The acquirer circuit is used to receive the other software code unit and provide the other software code unit to the cache memory; The cache memory is used to store the other software code unit; and The processor unit is used to receive the other software code unit from the cache memory and execute the other software code unit.

9. The device according to claim 7, wherein, The acquirer circuitry is configured to provide the first request to the remote device and, after failing to retrieve the other software code unit from the remote memory in response to the first request, provide the second request to the NVM controller.

10. The device according to claim 9, wherein, The acquirer circuit is configured to determine the failure of the first request based on at least one configured strategy, the at least one configured strategy including one or more of the following: a maximum number of retries to receive the other software code unit in response to the first request; a timeout period for waiting to receive the other software code unit in response to the first request; and identification of network problems. Identification of problems with the remote device; and identification of errors in the other software code unit received from the remote memory.

11. The device according to claim 6, wherein, The acquirer circuit is used to execute acquirer software code that performs cache replacement only from the local NVM.

12. The device according to claim 1, wherein, The network device is a network switch.

13. The device according to claim 1, wherein, The remote memory is dynamic random access memory (DRAM).

14. The device according to claim 1, wherein, The acquirer circuit is used to receive packets containing the given software code unit out of order.

15. The device according to claim 14, wherein, The given software code unit is an authenticable software code unit, and the acquirer circuit is configured to authenticate the given software code unit after receiving all packets of a given request from the remote memory.

16. The device according to claim 1, wherein, The acquirer circuitry is used to receive the given software code unit from the remote memory via an unreliable connection to the network.

17. The device according to claim 1, further comprising: Read-only memory (ROM) contains boot management and recovery routines.

18. The device according to claim 1, wherein, The acquirer circuitry is configured to restrict the reception of the given software code unit to at least one of the following: a given network; or a given port connection of the remote device.

19. The device according to claim 1, wherein, The acquirer circuit is used to receive the given software code unit in encrypted form from the remote device.

20. The device according to claim 1, wherein, The acquirer circuit is configured to: receive the given software code unit in unencrypted form from the remote device, and authenticate the given software code unit before executing it.

21. The device according to claim 1, wherein, The acquirer circuitry is used to attempt to retrieve another software code unit from multiple remote memories on multiple corresponding remote devices via the network as part of the cache replacement process.

22. A method comprising: Obtain software code units of the software code and load them into the cache memory; The given software code unit that caused the cache miss is checked in the cache memory; The given software code unit is received via a network from a remote memory located in a remote device as part of the cache replacement process; as well as The given software code unit is provided to the cache memory.

Images