Network controller, electronic device, computer network system and remote access method

Abstract

The embodiment of the invention discloses a network controller, electronic equipment, a computer network system and a remote access method, relates to the technical field of integrated computers, and can effectively simplify the access operation of a processor on a remote memory. The network controller comprises a semantic conversion module which is configured to convert a memory access request into a network access request under network semantics under the condition that a processor initiates the memory access request to a first memory address; wherein the first memory address is an address within a first address range in physical memory addresses of first equipment; and the control module is electrically connected with the semantic conversion module and is configured to send the network access request to the second equipment, so that the memory access request acts on a memory buffer area corresponding to a second memory address of the second equipment through the network access request, and the memory buffer area corresponds to the second memory address of the second equipment. The second memory address is located in the second address range. The method is suitable for remote memory access.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a network controller, electronic device, computer network system, and remote access method. Background Technology

[0002] With the development of computer and network technologies, remote memory access technology has become increasingly common. In remote memory access technology, one host can remotely access the memory of another host by calling a preset application programming interface (API).

[0003] However, since remote memory access requires calling the corresponding application programming interface, many operations based on remote memory access are quite cumbersome and differ significantly from local memory access. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a network controller, an electronic device, a computer network system, and a remote access method, which can effectively simplify the processor's access to remote memory.

[0005] In a first aspect, embodiments of the present invention provide a network controller, the network controller comprising: A semantic conversion module is configured to convert a memory access request into a network access request under network semantics when the processor initiates a memory access request to a first memory address; wherein the first memory address is an address within a first address range in the physical memory address of the first device; the semantic conversion module stores an address mapping relationship between the first address range and a second address range of the second device, wherein the first address range and the second address range are of equal size; a control module is electrically connected to the semantic conversion module and is configured to send the network access request to the second device so that the memory access request, through the network access request, acts on the memory buffer corresponding to the second memory address of the second device, wherein the second memory address is within the second address range.

[0006] In one embodiment, the network controller further includes: an address placeholder module configured to detect whether a memory access request initiated by the processor is a memory access request for the first memory address; and a semantic conversion module configured to convert the memory access request into a network access request under network semantics when the address placeholder module detects that the memory access request initiated by the processor is a memory access request for the first memory address.

[0007] In one implementation, the address placeholder module is further configured to obtain the first address range allocated by the basic input / output system or operating system before detecting whether the memory access request initiated by the processor is a memory access request for the first memory address.

[0008] In one embodiment, the semantic conversion module is further configured to: after the address placeholder module obtains the first address range allocated by the basic input / output system or operating system, and before the semantic conversion module converts the memory access request into a network access request under network semantics, determine the base address of the second address range and the size of the second address range based on the base address of the first address range and the size of the first address range; the control module is further configured to send the base address of the second address range and the size of the second address range to the second device, so that the second device establishes a memory buffer with the second address range.

[0009] In one embodiment, the control module is further configured to: after sending the base address of the second address range and the size of the second address range to the second device, receive a remote memory access key sent by the second device; and after the semantic conversion module converts the memory access request into the network access request, send the network access request and the remote memory access key to the second device.

[0010] In one implementation, the second address range is a virtual memory address range or a physical memory address range.

[0011] In one implementation, the processor runs at least one process, and the first address range corresponds to a target process in the at least one process; the first memory address is obtained by mapping the virtual memory address to be accessed by the processor in the target process via a memory management unit.

[0012] In one implementation, there are multiple first address ranges, and each first address range does not overlap with the others. Each first address range corresponds to a target process running in the processor. The second address ranges corresponding to each first address range may be the same or different. The semantic conversion module stores the address mapping relationship between each first address range and its corresponding second address range.

[0013] In one implementation, the second address range corresponds to a process running in the second device.

[0014] In one embodiment, the semantic conversion module stores the base address of the first address range, the size of the first address range, and the address mapping relationship between the base address of the first address range and the base address of the second address range of the second device.

[0015] In one implementation, the base address of the first address range is the same as the base address of the second address range of the second device.

[0016] In one embodiment, the memory access request includes: a first memory address to be accessed by the memory access request, and a first memory operation to be performed on the first memory address; the semantic conversion module is configured to, when the processor initiates a memory access request for the first memory address, determine the second memory address based on the first memory address and the address mapping relationship between the base address of the first address range and the base address of the second address range of the second device, and convert the first memory operation into a first network operation under network semantics; the control module is configured to encapsulate the second memory address and the first network operation into a network packet and send the network packet to the second device.

[0017] In one embodiment, the control module is further configured to receive a network response result from the second device to the network access request; the semantic conversion module is further configured to convert the network response result into a memory response result in memory semantics, and notify the processor of the memory response result.

[0018] In one implementation, the memory access request includes at least one of the following: a memory access request corresponding to a load command, a memory access request corresponding to a store command, and a memory access request corresponding to an atomic operation command.

[0019] In one implementation, the network access request includes at least one of the following: a network read request, a network read request with an immediate value, a network write request, a network write request with an immediate value, and a message sending request; the network access request is based on any of the following network protocols: Remote Direct Memory Access Protocol, Transmission Control Protocol / Internet Protocol, and Fibre Channel Storage Protocol.

[0020] Secondly, embodiments of the present invention also provide an electronic device, the electronic device including a processor and a network controller, the processor being electrically connected to the network controller, wherein the network controller is any of the network controllers provided in the foregoing embodiments.

[0021] Thirdly, embodiments of the present invention also provide a computer network system, including a first device and a second device, wherein the first device is any of the electronic devices provided in the foregoing embodiments, and the first device forms a network with the second device in a vertically extended manner through its own network controller.

[0022] Fourthly, embodiments of the present invention also provide a remote access method, the method comprising: when a processor initiates a memory access request to a first memory address, converting the memory access request into a network access request under network semantics; wherein, the first memory address is an address within a first address range in the physical memory address of a first device; a preset address mapping relationship exists between the first address range and a second address range of a second device, and the first address range and the second address range are of equal size; sending the network access request to the second device, so that the memory access request, through the network access request, acts on the memory buffer corresponding to the second memory address of the second device, the second memory address being within the second address range.

[0023] In one embodiment, the method further includes: detecting whether the memory access request initiated by the processor is a memory access request for the first memory address; the step of converting the memory access request into a network access request under network semantics when the processor initiates a memory access request for the first memory address includes: converting the memory access request into a network access request under network semantics when it is detected that the memory access request initiated by the processor is a memory access request for the first memory address.

[0024] In one implementation, before detecting whether the memory access request initiated by the processor is a memory access request for the first memory address, the method further includes: obtaining the first address range allocated by the basic input / output system or the operating system.

[0025] In one embodiment, the method further includes: after obtaining the first address range allocated by the basic input / output system or operating system, and before converting the memory access request into a network access request under network semantics, determining the base address of the second address range and the size of the second address range based on the base address of the first address range and the size of the first address range; and sending the base address of the second address range and the size of the second address range to the second device, so that the second device establishes a memory buffer with the second address range.

[0026] In one embodiment, the method further includes: after sending the base address of the second address range and the size of the second address range to the second device, receiving a remote memory access key sent by the second device; the step of sending the network access request to the second device includes: after converting the memory access request into the network access request, sending the network access request and the remote memory access key to the second device.

[0027] In one implementation, the second address range is a virtual memory address range or a physical memory address range.

[0028] In one implementation, the processor runs at least one process, and the first address range corresponds to a target process in the at least one process; the first memory address is obtained by mapping the virtual memory address to be accessed by the processor in the target process via a memory management unit.

[0029] In one implementation, there are multiple first address ranges, and each first address range does not overlap with the others. Each first address range corresponds to a target process running in the processor. The second address ranges corresponding to each first address range may be the same or different. There is a preset address mapping relationship between each first address range and its corresponding second address range.

[0030] In one implementation, the second address range corresponds to a process running in the second device.

[0031] In one embodiment, the first address range is determined by the base address of the first address range and the size of the first address range, and there is a preset address mapping relationship between the base address of the first address range and the base address of the second address range of the second device.

[0032] In one implementation, the base address of the first address range is the same as the base address of the second address range of the second device.

[0033] In one implementation, the memory access request includes: a first memory address to be accessed by the memory access request, and a first memory operation to be performed on the first memory address; the step of converting the memory access request into a network access request under network semantics when the processor initiates a memory access request for the first memory address includes: determining the second memory address based on the first memory address and the address mapping relationship between the base address of the first address range and the base address of the second address range of the second device, and converting the first memory operation into a first network operation under network semantics; the step of sending the network access request to the second device includes: encapsulating the second memory address and the first network operation into a network packet, and sending the network packet to the second device.

[0034] In one embodiment, the method further includes: receiving a network response result from the second device to the network access request; converting the network response result into a memory response result in memory semantics; and notifying the processor of the memory response result.

[0035] In one implementation, the memory access request includes at least one of the following: a memory access request corresponding to a load command, a memory access request corresponding to a store command, and a memory access request corresponding to an atomic operation command.

[0036] In one implementation, the network access request includes at least one of the following: a network read request, a network read request with an immediate value, a network write request, a network write request with an immediate value, and a message sending request; the network access request is based on any of the following network protocols: Remote Direct Memory Access Protocol, Transmission Control Protocol / Internet Protocol, and Fibre Channel Storage Protocol.

[0037] The network controller, electronic device, computer network system, and remote access method provided in this invention can convert a memory access request to a first memory address into a network access request under network semantics when the processor initiates a memory access request, and send the network access request to the second device. This allows the memory access request to act on the memory buffer corresponding to the second memory address of the second device through the network access request. Since the second address range of the second device and the first address range have an address mapping relationship and their sizes are equal, the memory buffer corresponding to the second address range in the second device can be mapped to the local address space of the first device. When the processor initiates an access request to the first memory address, since the first memory address is an address within the first address range of the first device's physical memory address, the access request to the first memory address can be mapped to the second memory address within the second address range based on the address mapping relationship between the first and second address ranges. This enables remote access to the memory of the second device. In other words, the processor does not perceive the difference between local and remote memory and can directly access remote memory as if it were local memory, without needing to call related application programming interfaces. Therefore, it effectively simplifies the processor's access operations to remote memory. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 A schematic diagram of a network controller provided for an embodiment of the present invention; Figure 2 A schematic diagram of another structure of a network controller provided for an embodiment of the present invention; Figure 3 This is a schematic diagram of the system environment in which the network controller is located in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating network communication performed by the network controller in a system environment, as described in an embodiment of the present invention. Figure 5 This is a schematic diagram of an address mapping relationship in an embodiment of the present invention; Figure 6 A schematic diagram of the structure of an electronic device provided for an embodiment of the present invention; Figure 7 A schematic diagram of a computer network system provided for an embodiment of the present invention; Figure 8 A flowchart of a remote access method provided for embodiments of the present invention; Figure 9 A detailed flowchart of a remote access method provided for an embodiment of the present invention. Detailed Implementation

[0040] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0041] Before describing the embodiments of this application, the relevant terms involved in this application will be explained.

[0042] RDMA: Remote Direct Memory Access, a mechanism that allows direct access to the memory of a remote host. WQE: Work Queue Element, an element in the RDMA work queue; SQ: Send Queue; RQ: Received Queue, the data receiving queue; QP: Queue Pair, a queue pair consisting of a data sending queue and a data receiving queue; CQ: Completion Queue; CQE: Completion Queue Element, an element in the work completion queue; WR: Work Request; WC: Work Completion; MR: Memory Region, the memory region used for RDMA data transmission and reception; MTT: Memory Translation Table, a table for translating virtual addresses to physical addresses; MPT: Memory Protection Table, a table that records access permissions for memory regions; MMU: Memory Management Unit; RNIC: RDMA Network Interface Controller; OPCODE: Operation Code.

[0043] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0044] In a first aspect, embodiments of the present invention provide a network controller that can effectively simplify the processor's access to remote memory.

[0045] like Figure 1 As shown, an embodiment of the present invention provides a network controller, the network controller comprising: The semantic conversion module 11 is configured to convert the memory access request into a network access request under network semantics when the processor initiates a memory access request to a first memory address; wherein, the first memory address is an address in the physical memory address of the first device that is within a first address range; the semantic conversion module 11 stores an address mapping relationship between the first address range and the second address range of the second device, wherein the first address range and the second address range are of equal size; The control module 12, electrically connected to the semantic conversion module 11, is configured to send the network access request to the second device so that the memory access request acts on the memory buffer corresponding to the second memory address of the second device through the network access request, wherein the second memory address is located within the second address range.

[0046] The network controller provided in this embodiment of the invention can convert a memory access request to a first memory address into a network access request under network semantics when the processor initiates a memory access request, and send the network access request to the second device so that the memory access request acts on the memory buffer corresponding to the second memory address of the second device through the network access request. Since the second address range of the second device and the first address range have an address mapping relationship and the address range sizes are equal, the memory buffer corresponding to the second address range in the second device can be mapped to the local address space of the first device. When the processor initiates an access request to the first memory address, since the first memory address is an address within the first address range of the physical memory address of the first device, according to the address mapping relationship between the first address range and the second address range, the access request to the first memory address can be mapped to the second memory address within the second address range, thereby realizing remote access to the memory of the second device. That is to say, the processor will not perceive the difference between local memory and remote memory, and can directly access remote memory as if it were local memory, without calling the relevant application programming interface, thus effectively simplifying the processor's access operation to remote memory.

[0047] In embodiments of the present invention, the network controller may include a device or circuit module for network communication control based on various network protocols, which can be electrically connected to the processor via a PCIe (Peripheral Component Interconnect Express) interface. The network controller may include a semantic translation module for converting a processor's memory access request for a first memory address within a first address range into a network access request for a second memory address within a second address range of a second device. Here, the processor may include one or more central processing units or coprocessors (e.g., GPUs). Since the processor can also be a GPU, GPU processes can directly push kernel functions to the GPU to access network memory with memory semantics, enhancing the GPU's ability to utilize the network.

[0048] For the processor, the first memory address is a segment of physical memory that can be accessed. However, the first memory address may not have a corresponding physical memory space on the first device. In other words, the first memory address is merely a physical address, and there may not be any physical memory with the first memory address in the first device. Here, the processor and network controller can be located in the first device.

[0049] Since the first memory address is an accessible physical memory address for the processor, the processor can initiate various memory access operations on the first memory address, such as reading memory and writing memory. When the processor initiates a memory access request on the first memory address, the semantic translation module can convert the memory access request into a network access request in network semantics.

[0050] Here, network semantics is a concept relative to memory semantics. Memory access under memory semantics can refer to the processor directly accessing memory through memory addresses. In one implementation, the memory access request may include at least one of the following: a memory access request corresponding to a load command, a memory access request corresponding to a store command, or a memory access request corresponding to an atomic operation command.

[0051] Network access in network semantics can refer to a processor accessing the network by calling an application programming interface (API), such as the Verbs interface of RDMA. In one implementation, the network access request may include one or more of the following: a network read request, a network read request with an immediate value, a network write request, a network write request with an immediate value, and a message transmission request.

[0052] The network access request can be based on various network protocols. For example, in one instance, the network access request can be based on any of the following network protocols: Remote Direct Memory Access Protocol, Transmission Control Protocol / Internet Protocol, or Fibre Channel Storage Protocol.

[0053] After the semantic translation module converts the memory access request for the first memory address into a network access request, the control module can send this network access request to the second device. This allows the memory access request to operate on the memory buffer corresponding to the second memory address on the second device through the network access request, such as performing read and write operations on the memory buffer at the second memory address. Thus, from the processor's perspective, the operation appears to be an access operation to local memory, but the actual operation is an access operation to remote memory.

[0054] Specifically, in one embodiment of the present invention, at least one process may run in the processor, and the processor's memory access requests may specifically include memory access requests made by the processor when running each process. A first address range may correspond to one of the target processes among the at least one process; that is, the first address range represents the resources owned by the first target process. In this way, different target processes can have their own address mapping relationships, thereby isolating the address mappings between the target processes and effectively improving security.

[0055] In one implementation, the processor may carry the virtual memory address to be accessed in the memory access request initiated when running the target process. The first memory address can be obtained by mapping the virtual memory address to be accessed by the processor in the target process through the memory management unit (MMU).

[0056] Optionally, in embodiments of the present invention, the number of first address ranges can be one or more. When there are multiple first address ranges, each first address range does not overlap, and each first address range corresponds to one of the target processes running in the processor. In embodiments of the present invention, depending on specific application requirements, the second address ranges corresponding to each first address range can be the same or different; the embodiments of the present invention do not limit this. Accordingly, the semantic conversion module 11 can store the address mapping relationship between each first address range and its corresponding second address range. For example, first address range A1 corresponds to second address range B1, first address range A2 corresponds to second address range B2, first address range A3 and first address range A4 both correspond to second address range B3, etc.

[0057] In embodiments of the present invention, the second address range in the second device can be either a virtual memory address range or a physical memory address range. When the second address range is a physical memory address range, the second memory address is also a physical memory address, and therefore can be directly accessed based on the memory buffer corresponding to the second memory address. When the second address range is a virtual memory address range, the second memory address is also a virtual memory address, and the second device can query the address mapping page table to convert the second memory address into the corresponding physical memory address, thereby accessing the corresponding memory buffer.

[0058] In one implementation, the second address range may correspond to a process running in the second device; that is, the second address range may be the internal address range of a process in the second device.

[0059] Furthermore, in one embodiment of the present invention, as Figure 2 As shown, the network controller may further include: an address placeholder module 13, configured to detect whether the memory access request initiated by the processor is a memory access request for the first memory address; based on this, the semantic conversion module 11 may be configured to convert the memory access request into a network access request under network semantics when the address placeholder module detects that the memory access request initiated by the processor is a memory access request for the first memory address.

[0060] In embodiments of the present invention, the address placeholder module 13 can be used as part of the PCIe interface for interaction between the network controller and the processor. Specifically, the processor can access local memory based on a memory address. If the address the processor wants to access is not within the first address range, it means the processor is accessing its local memory, and the address placeholder module does not need to trigger the semantic translation module to perform semantic translation. Conversely, if the address the processor wants to access is within the first address range, it means the processor is accessing remote memory, and the address placeholder module can trigger the semantic translation module to perform semantic translation.

[0061] In specific implementation, the address placeholder module 13 can be any device or circuit module that can occupy a range of physical addresses. For example, in one example, the address placeholder module 13 can be a base address register (BAR).

[0062] In one implementation, the address placeholder module 13 can acquire a first address range allocated by the Basic Input / Output System (BIOS) or the Operating System (OS) before detecting whether the memory access request initiated by the processor is a memory access request for the first memory address. This allows for subsequent processing to determine whether the memory address to be accessed by the processor is a first memory address within the first address range. For example, in one instance, the address placeholder module 13 can acquire the first address range allocated by the BIOS when the first device powers on. In another instance, the address placeholder module 13 can acquire the first address range allocated by the operating system after the first device powers on.

[0063] In embodiments of the present invention, the network controller can be based on various network protocols. Different network protocols result in different networking methods for the first and second devices, as well as different forms of network access requests. The semantic conversion module can convert memory access requests into corresponding network access requests according to the specific network protocol. The control module can encapsulate the corresponding network requests into network packets according to the specific network protocol and send them to the second device.

[0064] The following section uses the RDMA network protocol as an example to briefly introduce the system environment in which the network controller is located.

[0065] like Figure 3 As shown, the first and second devices are connected via an RDMA network, where the RNIC is a network controller based on the RDMA protocol. The RNIC network controller includes a PCIe module, which can be electrically connected to the processor via a PCIe switch (PCIeswitch). The RNIC network controller may include an address placeholder module, a semantic translation module, and a control module. The address placeholder module can be a component of the PCIe module.

[0066] The address placeholder module can occupy a first address range. When the processor initiates a memory access request for the first address range using memory semantics instructions such as load, store, and atomic, the semantic conversion module can convert the memory access request into a network access request under network semantics. The control module can encapsulate this network access request into an RDMA message and send it to the second device to access the memory buffer (e.g., the memory buffer corresponding to the second memory address) in the second device. Figure 3 (BUF in the middle).

[0067] The embodiments provided by this invention convert the memory access request into a network access request under network semantics. In this way, the local CPU / GPU can directly access the remote memory through instructions such as Load / Store, which can shield memory differences and access the remote memory directly as if it were local memory. This simplifies the operation of remote memory. Moreover, the processor directly accesses the memory of the remote host, eliminating additional data copying and improving data transmission efficiency.

[0068] In one implementation, the semantic conversion module is further configured to: after the address placeholder module obtains the first address range allocated by the basic input / output system or operating system, and before the semantic conversion module converts the memory access request into a network access request under network semantics, determine the base address and size of the second address range based on the base address and size of the first address range; the control module is further configured to send the base address and size of the second address range to the second device, so that the second device establishes a memory buffer with the second address range. In this way, the first address range and the second address range can be mapped using the base address and the size of the address range.

[0069] In one implementation, the control module can further receive a remote memory access key sent by the second device after sending the base address of the second address range and the size of the second address range to the second device. Based on this, the control module can also send the network access request and the remote memory access key to the second device after the semantic conversion module converts the memory access request into the network access request. The second device can authenticate the remote memory access key, and only network access requests that pass authentication are allowed to access the second memory address of the second device.

[0070] Taking the network controller under the RDMA protocol as an example, such as Figure 4 As shown, in one embodiment of the present invention, after entering the system, the RDMA communication connection and buffer network mapping relationship can be established by following the steps below.

[0071] Step 1: The control process of the second device listens to the preset port corresponding to the control process and waits for the first device to establish a connection RDMA connection; the control process of the first device initiates a connection establishment to the second device, and the RDMA connection is successfully established; the connection-related SQ / RQ / CQ queue buffer resources are all created from the on-chip memory of the RNIC; The control process of the second device can refer to a process running on the second device that is remotely accessed by the memory of the first device. The control process of the first device can refer to a process running on the first device that needs to remotely access the memory of the second device.

[0072] Step 2: The first device sends the base address value and size of the BAR space to the second device via a network connection; Among them, the BAR space is the first address range.

[0073] Step 3: The second device receives the base address and size of the BAR space sent by the first device, uses the base address value as the process virtual address value, and allocates a buffer of the same size as the BAR space for mapping with the BAR space of the first device.

[0074] Step 4: After the first device establishes an RDMA communication connection and completes the buffer network mapping, the second device's control process can register the buffer with its local RNIC. Specifically, during memory registration, the RNIC chip's internal memory creates a Memory Access Table (MPT) to record the access permissions of the registered memory. Simultaneously, it establishes an MTT table to translate the virtual address of the process buffer into a physical address, thereby establishing the network mapping relationship between the first device's BAR space and the second device's process buffer. Upon successful registration, the RNIC returns two keys to the control process: a remote key (rkey) and a local key (lkey). The second device's control process sends the rkey to the first device for subsequent memory semantic access. Step 5: After receiving the rkey, the first device sends the base addresses and read / write pointer information of the SQ, RQ, and CQ queues of the RDMA connection, as well as the access rkey of the second device's process buffer, to the semantic conversion module. Upon receiving the above information, the semantic conversion module completes the configuration and enters the ready state. At this point, initialization is complete.

[0075] The above embodiments focus on how to establish an RDMA communication connection. After establishing the RDMA communication connection, the semantic conversion module can be used to map the remote process buffer to the local BAR space. In one embodiment, the semantic conversion module 11 stores the base address of the first address range, the size of the first address range, and the address mapping relationship between the base address of the first address range and the base address of the second address range of the second device.

[0076] Specifically, the base address of the first address range can be the same as or different from the base address of the second address range of the second device. In different cases, the base address of the first address range can be converted to the base address of the second address range of the second device. For example, if the first device address is 0x1234 and the second device address is 0x4567, a conversion table can be added to the semantic conversion module to convert 0x1234 to 0x4567. Thus, when the semantic conversion module receives an address request with address 0x1234, it can look up the conversion table and convert 0x1234 to 0x4567. Subsequently, when filling the WQE in the transmit queue SQ of the RDMA network controller, the address filled in is 0x4567.

[0077] For example, the address mapping relationship between the first device and the second device can be as follows: Figure 5 As shown.

[0078] After the network mapping relationship is established, the processor can initiate memory access to the process's virtual address space through a network access request. In one embodiment, the memory access request may include: the first memory address to be accessed by the memory access request, and a first memory operation to be performed on the first memory address; the semantic conversion module may be configured to, when the processor initiates a memory access request for the first memory address, determine the second memory address based on the first memory address and the address mapping relationship between the base address of the first address range and the base address of the second address range of the second device, and convert the first memory operation into a first network operation under network semantics; the control module may be configured to encapsulate the second memory address and the first network operation into a network packet and send the network packet to the second device.

[0079] In one embodiment, the control module 12 may further be configured to receive the network response result of the second device to the network access request; the semantic conversion module 11 is further configured to convert the network response result into a memory response result in memory semantics, and notify the processor of the memory response result. In this embodiment, the control module receives feedback on the result of remote access, and then the semantic conversion module converts the network response result into a memory response result in memory semantics and notifies the processor.

[0080] The following section uses an RDMA network as an example to explain write access and read access to remote memory.

[0081] Please see again Figure 4In one embodiment of the present invention, when the processor issues a store instruction to write to the process virtual address space, the write memory semantic translation process is as follows: Step 6: The processor issues a store instruction. The virtual address operated on by the instruction is translated by the MMU, and the physical address falls into the BAR space. Among them, the BAR space is the first address range.

[0082] Step 7: Notify the semantic conversion module of the message to be written to the BAR space, and at the same time synchronize the address of the BAR space to be written and the written value to the semantic conversion module. Step 8: The semantic translation module fills the SQ queue of the RDMA connection with WQE, fills the write address into the VA field, fills the write value into the immediate number field, and fills the OPCODE field with the RDMA immediate number write; at the same time, the rkey for accessing the peer memory is also filled into WQE. After the WQE is filled, the semantic translation module writes to the SQ queue doorbell register. Step 9: After receiving the doorbell message, the control module processes the WQE of the SQ queue, parses the OPCODE as RDMA immediate write, and encapsulates the address rvaddr of the immediate write operation, the value data, the data length (len), and the remote memory access key rkey into an RDMA network packet. Step 10: Send the RDMA network message to the second device; Step 11: The network controller of the second device receives and processes the network packet, parses the RDMA OPCODE as an immediate write, extracts the memory access virtual address (the BAR space address of the first device) from the packet, queries the MPT table for authentication, and after successful authentication, queries the MTT table to convert the virtual address to the physical address (PA). Step 12: The network controller of the second device writes the immediate value into the process buffer (i.e., Dst Buff). Step 13: After the network controller of the second device finishes writing the data, it replies to the first device that the write was successful; the first device writes the write completion message to the CQ queue via CQE. Step 14: The semantic conversion module of the first device reads the CQ completion message and parses the CQE; Step 15: The semantic translation module of the first device replies with a success message to the PCIe module; Step 16: The PCIe module replies with a completion message to the processor (CPU or GPU). This completes the write operation to the remote network memory.

[0083] See also Figure 4In another embodiment of the present invention, when the processor issues a load instruction, i.e., a memory access request corresponding to a storage command, to read the process virtual address space, the specific implementation of the memory read semantic translation process may include the following: Step 6: The processor issues a load instruction. The virtual address operated on by the instruction is translated by the MMU, and the physical address falls into the BAR space. Step 7: Read the message notification from the BAR space to the semantic conversion module, and simultaneously synchronize the address of the write BAR space to the semantic conversion module; Step 8: The semantic translation module fills the WQE into the SQ queue of the RDMA connection, filling the read address into the VA field, the read data size into the len field, and the OPCODE field with RDMA immediate read (new); at the same time, the rkey for accessing the peer's memory is also filled into the WQE. After completing the filling of the WQE, the semantic translation module writes to the SQ queue doorbell register; Step 9: After receiving the doorbell message, the control module processes the WQE of the SQ queue, parses the OPCODE as RDMA immediate read, and encapsulates the address of the immediate read operation, the size of the read data, and the rkey of the remote memory access into an RDMA network packet. Step 10: Send the RDMA network message to the second device; Step 11: The network controller of the second device receives and processes the network packet, parses the RDMA OPCODE as an immediate read, extracts the memory access virtual address (the BAR space address of the first device) in the packet, queries the MPT table for authentication, and after successful authentication, queries the MTT table to convert the virtual address to the physical address (PA). Step 12: Read the immediate value from the process buffer; Step 13: After the network controller of the second device reads the data, it encapsulates the data into an RDMA network packet and replies to the first device; the first device writes the read data into the immediate digital segment of the CQE and pushes it to the CQ queue. Step 14: The semantic conversion module of the first device reads the CQ completion message and parses the CQE; Step 15: The semantic translation module of the first device retrieves the immediate value from the CQE and sends it to the PCIe module; Step 16: The PCIe module sends the read data to the processor (CPU or GPU).

[0084] This completes the read operation on the remote network memory.

[0085] The above describes the entire process of load / store memory semantic conversion. The conversion process for other atomic operation semantics is similar and will not be repeated here. The correspondence between memory semantics and network semantics is shown in Table 1. For different semantic conversions, the semantic processing module is responsible for filling in the WQE of the SQ and reading the CQE of the CQ, and performing the corresponding parsing.

[0086]

[0087] Secondly, embodiments of the present invention also provide an electronic device that can effectively simplify the processor's access to remote memory.

[0088] like Figure 6 As shown, the electronic device 7 provided in the embodiment of the present invention includes a processor 71 and a network controller 72, wherein the processor 71 and the network controller 72 are electrically connected; wherein the network controller 72 is any of the network controllers provided in the foregoing embodiments, and thus can also achieve the corresponding beneficial technical effects, which have been described in detail above and will not be repeated here.

[0089] Thirdly, such as Figure 7 As shown, embodiments of the present invention also provide a computer network system 80, including a first device 81 and a second device 82. The first device 81 can be any type of electronic device provided in the embodiments of the present invention. The first device 81 and the second device 82 are networked in a vertically extended manner through the first device's own network controller, thus achieving the corresponding technical effects. This has been described in detail above and will not be repeated here. In the embodiments of the present invention, since the first device 81 can remotely access the memory of the second device 82 using memory semantics through its own network controller, the first device 81 and the second device 82 can be networked in a vertically extended manner, resulting in a more flexible and diverse network configuration and better network performance.

[0090] Fourthly, embodiments of the present invention also provide a remote access method that can effectively simplify the processor's access operations to remote memory.

[0091] like Figure 8 As shown, the remote access method provided by the embodiments of the present invention may include: S91, when the processor initiates a memory access request to the first memory address, the memory access request is converted into a network access request under network semantics; wherein, the first memory address is an address within a first address range in the physical memory address of the first device; there is a preset address mapping relationship between the first address range and the second address range of the second device, and the first address range and the second address range are of equal size; S92, the network access request is sent to the second device so that the memory access request acts on the memory buffer corresponding to the second memory address of the second device through the network access request, and the second memory address is located within the second address range.

[0092] The remote access method provided in this invention can convert a memory access request to a first memory address into a network access request under network semantics when the processor initiates a memory access request, and send the network access request to the second device. This allows the memory access request to act on the memory buffer corresponding to the second memory address of the second device through the network access request. Since the second address range of the second device and the first address range have an address mapping relationship and their sizes are equal, the memory buffer corresponding to the second address range in the second device can be mapped to the local address space of the first device. When the processor initiates an access request to the first memory address, since the first memory address is an address within the first address range of the physical memory address of the first device, the access request to the first memory address can be mapped to the second memory address within the second address range based on the address mapping relationship between the first and second address ranges. This enables remote access to the memory of the second device. In other words, the processor does not perceive the difference between local and remote memory and can directly access remote memory as if it were local memory, without needing to call related application programming interfaces. Therefore, it effectively simplifies the processor's access operations to remote memory.

[0093] In one embodiment, the method further includes: detecting whether the memory access request initiated by the processor is a memory access request for the first memory address; the step of converting the memory access request into a network access request under network semantics when the processor initiates a memory access request for the first memory address includes: converting the memory access request into a network access request under network semantics when it is detected that the memory access request initiated by the processor is a memory access request for the first memory address.

[0094] In one implementation, before detecting whether the memory access request initiated by the processor is a memory access request for the first memory address, the method further includes: obtaining the first address range allocated by the basic input / output system or the operating system.

[0095] In one embodiment, the method further includes: after obtaining the first address range allocated by the basic input / output system or operating system, and before converting the memory access request into a network access request under network semantics, determining the base address of the second address range and the size of the second address range based on the base address of the first address range and the size of the first address range; and sending the base address of the second address range and the size of the second address range to the second device, so that the second device establishes a memory buffer with the second address range.

[0096] In one embodiment, the method further includes: after sending the base address of the second address range and the size of the second address range to the second device, receiving a remote memory access key sent by the second device; the step of sending the network access request to the second device includes: after converting the memory access request into the network access request, sending the network access request and the remote memory access key to the second device.

[0097] In one implementation, the second address range is a virtual memory address range or a physical memory address range.

[0098] In one implementation, the processor runs at least one process, and the first address range corresponds to a target process in the at least one process; the first memory address is obtained by mapping the virtual memory address to be accessed by the processor in the target process via a memory management unit.

[0099] In one implementation, there are multiple first address ranges, and each first address range does not overlap with the others. Each first address range corresponds to a target process running in the processor. The second address ranges corresponding to each first address range may be the same or different. There is a preset address mapping relationship between each first address range and its corresponding second address range.

[0100] In one implementation, the second address range corresponds to a process running in the second device.

[0101] In one embodiment, the first address range is determined by the base address of the first address range and the size of the first address range, and there is a preset address mapping relationship between the base address of the first address range and the base address of the second address range of the second device.

[0102] In one implementation, the base address of the first address range is the same as the base address of the second address range of the second device.

[0103] In one implementation, the memory access request includes: a first memory address to be accessed by the memory access request, and a first memory operation to be performed on the first memory address; the step of converting the memory access request into a network access request under network semantics when the processor initiates a memory access request for the first memory address includes: determining the second memory address based on the first memory address and the address mapping relationship between the base address of the first address range and the base address of the second address range of the second device, and converting the first memory operation into a first network operation under network semantics; the step of sending the network access request to the second device includes: encapsulating the second memory address and the first network operation into a network packet, and sending the network packet to the second device.

[0104] In one embodiment, the method further includes: receiving a network response result from the second device to the network access request; converting the network response result into a memory response result in memory semantics; and notifying the processor of the memory response result.

[0105] In one implementation, the memory access request includes at least one of the following: a memory access request corresponding to a load command, a memory access request corresponding to a store command, and a memory access request corresponding to an atomic operation command.

[0106] In one implementation, the network access request includes at least one of the following: a network read request, a network read request with an immediate value, a network write request, a network write request with an immediate value, and a message sending request; the network access request is based on any of the following network protocols: Remote Direct Memory Access Protocol, Transmission Control Protocol / Internet Protocol, and Fibre Channel Storage Protocol.

[0107] The remote access method provided by the embodiments of the present invention will be described in detail below through specific examples.

[0108] like Figure 9 As shown, the remote access method provided by the embodiments of the present invention may include: S501, Obtain the first address range allocated by the basic input / output system or operating system; S502. Determine the base address of the second address range and the size of the second address range based on the base address of the first address range and the size of the first address range; Optionally, the second address range is a virtual memory address range or a physical memory address range; Optionally, the base address of the first address range is the same as the base address of the second address range of the second device; S503, Send the base address of the second address range and the size of the second address range to the second device, so that the second device establishes a memory buffer with the second address range; S504. Receive the remote memory access key sent by the second device; S505: A memory access request was detected from the processor to the first memory address; wherein, the first memory address is an address within the first address range of the physical memory address of the first device; S506. Convert the memory access request into a network access request under network semantics; S507. Send the network access request along with the remote memory access key to the second device; S508. Receive the network response result from the second device to the network access request; S509. Convert the network response result into a memory response result in memory semantics, and notify the processor of the memory response result.

[0109] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0110] The various embodiments in this specification are described in a related manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0111] In particular, the device embodiment is basically similar to the method embodiment, so the description is relatively simple. For relevant details, please refer to the description of the method embodiment.

[0112] For ease of description, the above apparatus is described by dividing it into various functional units / modules. Of course, in implementing this invention, the functions of each unit / module can be implemented in one or more software and / or hardware.

[0113] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0114] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A network controller, characterized in that, include: The semantic conversion module is configured to convert a memory access request into a network access request under network semantics when the processor initiates a memory access request to a first memory address; wherein, the first memory address is an address within a first address range in the physical memory address of the first device; the semantic conversion module stores an address mapping relationship between the first address range and a second address range of the second device, wherein the first address range and the second address range are of equal size; The control module, electrically connected to the semantic conversion module, is configured to send the network access request to the second device so that the memory access request, through the network access request, acts on the memory buffer corresponding to the second memory address of the second device, wherein the second memory address is within the second address range.

2. The network controller according to claim 1, characterized in that, Also includes: The address placeholder module is configured to detect whether the memory access request initiated by the processor is a memory access request for the first memory address; The semantic conversion module is configured to convert the memory access request into a network access request under network semantics when the address placeholder module detects that the memory access request initiated by the processor is a memory access request for the first memory address.

3. The network controller according to claim 2, characterized in that, The address placeholder module is further configured to obtain the first address range allocated by the basic input / output system or operating system before detecting whether the memory access request initiated by the processor is a memory access request for the first memory address.

4. The network controller according to claim 3, characterized in that, The semantic conversion module is also configured to: After the address placeholder module obtains the first address range allocated by the basic input / output system or operating system, and before the semantic conversion module converts the memory access request into a network access request under network semantics, the base address of the second address range and the size of the second address range are determined based on the base address of the first address range and the size of the first address range. The control module is further configured to send the base address of the second address range and the size of the second address range to the second device, so that the second device establishes a memory buffer with the second address range.

5. The network controller according to claim 4, characterized in that, The control module is also configured to: After sending the base address of the second address range and the size of the second address range to the second device, the remote memory access key sent by the second device is received. After the semantic conversion module converts the memory access request into the network access request, it sends the network access request and the remote memory access key to the second device.

6. The network controller according to claim 1, characterized in that, The second address range is either the virtual memory address range or the physical memory address range.

7. The network controller according to claim 1, characterized in that, The processor runs at least one process, and the first address range corresponds to a target process in the at least one process; the first memory address is obtained by mapping the virtual memory address to be accessed by the processor in the target process through a memory management unit.

8. The network controller according to claim 7, characterized in that, There are multiple first address ranges, and each first address range does not overlap with the others. Each first address range corresponds to a target process running in the processor. The second address ranges corresponding to each first address range may be the same or different. The semantic conversion module stores the address mapping relationship between each first address range and the corresponding second address range.

9. The network controller according to claim 1, characterized in that, The second address range corresponds to the process running in the second device.

10. The network controller according to claim 1, characterized in that, The semantic conversion module stores the base address of the first address range, the size of the first address range, and the address mapping relationship between the base address of the first address range and the base address of the second address range of the second device.

11. The network controller according to claim 10, characterized in that, The base address of the first address range is the same as the base address of the second address range of the second device.

12. The network controller according to claim 10, characterized in that, The memory access request includes: the first memory address to be accessed by the memory access request, and the first memory operation to be performed on the first memory address; The semantic conversion module is configured to, when the processor initiates a memory access request to the first memory address, determine the second memory address based on the first memory address and the address mapping relationship between the base address of the first address range and the base address of the second address range of the second device, and convert the first memory operation into a first network operation under network semantics. The control module is configured to encapsulate the second memory address and the first network operation into a network packet and send the network packet to the second device.

13. The network controller according to any one of claims 1 to 12, characterized in that, The control module is also configured to receive the network response result of the second device to the network access request; The semantic conversion module is further configured to convert the network response result into a memory response result in memory semantics, and to notify the processor of the memory response result.

14. The network controller according to any one of claims 1 to 12, characterized in that, The memory access request includes at least one of the following: a memory access request corresponding to a load command, a memory access request corresponding to a store command, and a memory access request corresponding to an atomic operation command.

15. The network controller according to any one of claims 1 to 12, characterized in that, The network access request includes at least one of the following: network read request, network read request with immediate value, network write request, network write request with immediate value, and information sending request; the network access request is based on any of the following network protocols: Remote Direct Memory Access Protocol, Transmission Control Protocol / Internet Protocol, and Fibre Channel Storage Protocol.

16. An electronic device, characterized in that, The electronic device includes a processor and a network controller, the processor being electrically connected to the network controller, wherein the network controller is the network controller according to any one of claims 1-15.

17. A computer network system, characterized in that, It includes a first device and a second device, wherein the first device is any of the electronic devices described in claim 16, and the first device is networked with the second device in a vertically extended manner through its own network controller.

18. A remote access method, characterized in that, include: When the processor initiates a memory access request to the first memory address, the memory access request is converted into a network access request under network semantics; wherein, the first memory address is an address within a first address range in the physical memory address of the first device; there is a preset address mapping relationship between the first address range and the second address range of the second device, and the first address range and the second address range are of equal size; The network access request is sent to the second device so that the memory access request acts on the memory buffer corresponding to the second memory address of the second device through the network access request, and the second memory address is located within the second address range.

19. The method according to claim 18, characterized in that, Also includes: Detect whether the memory access request initiated by the processor is a memory access request for the first memory address; When the processor initiates a memory access request for a first memory address, converting the memory access request into a network access request under network semantics includes: If it is detected that the memory access request initiated by the processor is a memory access request for the first memory address, the memory access request is converted into a network access request under network semantics.

20. The method according to claim 19, characterized in that, Before detecting whether the memory access request initiated by the processor is a memory access request for the first memory address, the method further includes: obtaining the first address range allocated by the basic input / output system or the operating system.

21. The method according to claim 20, characterized in that, Also includes: After obtaining the first address range allocated by the basic input / output system or operating system, and before converting the memory access request into a network access request under network semantics, the base address of the second address range and the size of the second address range are determined based on the base address of the first address range and the size of the first address range. The base address of the second address range and the size of the second address range are sent to the second device so that the second device can establish a memory buffer with the second address range.

22. The method according to claim 21, characterized in that, Also includes: After sending the base address of the second address range and the size of the second address range to the second device, the remote memory access key sent by the second device is received. Sending the network access request to the second device includes: After converting the memory access request into the network access request, the network access request and the remote memory access key are sent to the second device.

23. The method according to claim 18, characterized in that, The second address range is either the virtual memory address range or the physical memory address range.

24. The method according to claim 18, characterized in that, The processor runs at least one process, and the first address range corresponds to a target process in the at least one process; the first memory address is obtained by mapping the virtual memory address to be accessed by the processor in the target process through a memory management unit.

25. The method according to claim 24, characterized in that, There are multiple first address ranges, and each first address range does not overlap with the others. Each first address range corresponds to a target process running in the processor. The second address ranges corresponding to each first address range may be the same or different. There is a preset address mapping relationship between each of the first address ranges and the corresponding second address ranges.

Images