Memory access method, memory management unit, chip, device and storage medium

Abstract

The application provides a memory access method, a memory management unit, a chip, a device and a storage medium. A plurality of memory management units are included in a processing chip, each memory management unit includes a first address translation cache unit, and the processing chip includes a second address translation cache unit shared by the plurality of memory management units. A target memory management unit receives a data write request including a virtual memory address sent by a target external device, and if an address translation table item corresponding to the virtual memory address is not found in the target first address translation cache unit and the second address translation cache unit, a page table is queried, the address translation table item queried from the page table is stored in the target first address translation cache unit and the second address translation cache unit, respectively, and data is written into a corresponding physical memory address. By increasing the number of address translation cache units available for each memory management unit, the number of page table traversals can be reduced, and IO performance can be improved.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, and in particular to a memory access method, a memory management unit, a chip, a device, and a storage medium. Background Technology

[0002] The Memory Management Unit (MMU) is used to translate the virtual memory addresses accessed by the CPU core into actual physical memory addresses. When transferring data between I / O devices (usually referred to as peripherals, i.e., external devices) and memory, to improve data transfer rates and reduce CPU load, Direct Memory Access (DMA) technology can be used to delegate data transfer between peripherals and memory to the DMA controller. In this DMA scenario, the required memory management unit is the Input / Output Memory Management Unit (IOMMU). Its function, simply put, is to translate the virtual memory addresses accessed by the peripherals into physical memory addresses. Specifically, the IOMMU, based on DMA remapping, obtains the address translation relationship from the peripheral's virtual memory address (IO Virtual Address, IOVA) to its physical memory address (Hardware Physical Address / Host Physical Address, HPA) by traversing the page table. However, page table traversal typically takes hundreds of clock cycles, increasing the latency of DMA requests. To further optimize performance, an IOMMU introduces an IOTLB (IO Translation Lookaside Buffer) unit. This unit is designed to improve the speed of virtual memory address to physical memory address translation. It caches some frequently accessed page table contents, i.e., caches some address translation table entries (IOTLB entries) that have been queried, thereby improving I / O efficiency.

[0003] In practice, for example, a server often contains multiple IOMMUs, each with its own IOTLB. An IOMMU can only access its own IOTLB and cannot access the IOTLBs of other IOMMUs across the bus. In real-world applications, the following scenario might occur: multiple peripherals are connected to IOMMU1, while only one peripheral or no peripherals are connected to IOMMU2. In this case, because all peripherals connected to IOMMU1 consume IOTLB1, IOTLB1 will be quickly exhausted, meaning it will soon be full, requiring frequent page table lookups. Meanwhile, IOTLB2 in IOMMU2 might be idle, resulting in very low overall IOTLB utilization. Summary of the Invention

[0004] This invention provides a memory access method, a memory management unit, a chip, a device, and a storage medium to reduce page table traversal counts and improve I / O performance.

[0005] In a first aspect, embodiments of the present invention provide a memory access method applied to a target memory management unit included in a processing chip, wherein the target memory management unit is any one of at least one memory management unit included in the processing chip, and each memory management unit includes a first address translation cache unit; the method includes:

[0006] Receive a data write request sent by a target external device, the data write request including a virtual memory address, the target external device being connected to the target memory management unit;

[0007] If no address translation table entry corresponding to the virtual memory address is found in the first target address translation cache unit, then the second address translation cache unit is queried to see if the address translation table entry is contained; wherein, the address translation table entry contains the physical memory address corresponding to the virtual memory address, the second address translation cache unit is connected to the at least one memory management unit, and the first target address translation cache unit is the first address translation cache unit in the target memory management unit;

[0008] If the address translation table entry is not found in the second address translation cache unit, the page table is queried to determine the address translation table entry corresponding to the virtual memory address;

[0009] The address translation table entries retrieved from the page table are respectively stored in the target first address translation cache unit and the second address translation cache unit;

[0010] Write the data corresponding to the data write request to the physical memory address.

[0011] Secondly, embodiments of the present invention provide a memory management unit, including:

[0012] A first address translation cache unit and an address translation processing unit; wherein, the address translation processing unit is connected to a second address cache unit, the second address cache unit is connected to different address translation processing units located within the same processing chip, and the different address translation processing units are located in different memory management units within the processing chip;

[0013] The address translation processing unit is used to execute the memory access method as described in the first aspect.

[0014] Thirdly, embodiments of the present invention provide a processing chip, comprising:

[0015] At least one memory management unit as described in the second aspect, and a second address translation cache unit;

[0016] The second address cache unit is connected to each memory management unit.

[0017] Fourthly, embodiments of the present invention provide an electronic device, including: at least one processing chip as described in the third aspect, memory, and external devices.

[0018] Fifthly, embodiments of the present invention provide a non-transitory machine-readable storage medium storing executable code, wherein when the executable code is executed by a memory management unit in an electronic device as described in the fourth aspect, the memory management unit performs the memory access method as described in the first aspect.

[0019] In the solution provided in this embodiment of the invention, for example, a server is provided with multiple processor chips (such as CPU chips), each processor chip contains multiple memory management units (such as multiple IOMMUs), and each memory management unit includes a first address translation cache unit (such as a first IOTLB). Based on this, an additional second address translation cache unit (such as a second IOTLB) is provided in the processor chip for sharing among the multiple memory management units. This expands the number of available IOTLBs for the peripherals, thereby reducing the number of page table traversals.

[0020] Specifically, based on the aforementioned processor chip architecture, when the target memory management unit receives a data write request from its connected external device, it first sequentially checks the first and second address translation cache units to see if an address translation table entry corresponding to the IOVA carried therein exists. If not, it then performs a page table lookup. After obtaining the address translation table entry corresponding to the IOVA from the page table, it performs the corresponding data write-to-memory processing based on the HPA contained in the address translation table entry. Simultaneously, it stores the address translation table entry in both the first and second address translation cache units. This way, the next time address translation of the IOVA is needed, the address translation can be quickly completed based on the cached results of the address translation table entry in the first and second address translation cache units, reducing the number of page table traversals and improving I / O performance. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A schematic diagram illustrating the principle of peripherals accessing memory through the IOMMU;

[0023] Figure 2 This is a schematic diagram of the composition of a processing chip provided in an embodiment of the present invention;

[0024] Figure 3 A flowchart of a memory access method provided in an embodiment of the present invention;

[0025] Figure 4 A flowchart of a memory access method provided in an embodiment of the present invention;

[0026] Figure 5 This is an application diagram of a memory access method provided in an embodiment of the present invention;

[0027] Figure 6 This is an application diagram of a memory access method provided in an embodiment of the present invention;

[0028] Figure 7 This is a schematic diagram of the structure of an electronic device provided in this embodiment. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0031] First, a brief explanation of some technical terms involved in the embodiments of the present invention will be given.

[0032] IOMMU: Input / Output Memory Management Unit, a memory management unit used to translate virtual memory addresses to physical memory addresses for peripheral access.

[0033] IOVA: IO Virtual Address, virtual memory address.

[0034] HPA: Hardware Physical Address or Host Physical Address, physical memory address.

[0035] Peripherals: Short for external devices, which are connected to the IOMMU to access memory through the IOMMU.

[0036] Root Port: The root port is the interface provided by the processing chip (such as the CPU chip) for connecting peripherals, located in the root complex. It typically supports a certain bus protocol, such as the PCI (Peripheral Component Interconnect) protocol.

[0037] Page table: The operating system maintains a corresponding page table for each process / virtual machine (VM) to store the mapping between virtual memory addresses and physical memory addresses. Different processes / VMs can access isolated physical memory addresses, but they can access the same virtual memory addresses. Because different processes / VMs have different page tables, even when accessing the same virtual memory address, the translated physical memory address will be different.

[0038] IOTLB: IO Translation Lookaside Buffer, is a bypass lookaside buffer / address translation buffer unit used to cache the mapping relationship between the virtual memory address and the physical memory address recently translated by the IOMMU, that is, the address translation table entry (IOTLB entry) describing the mapping relationship.

[0039] PF: Physical Function, which simply means physical peripheral.

[0040] VF: Virtual Function. Simply put, it refers to a virtual peripheral device. It can be achieved by using a virtualization technology, such as SR-IOV (Single Root I / O Virtualization), to virtualize a physical peripheral device and obtain one or more corresponding virtual peripheral devices, which can then be allocated to one or more virtual machines.

[0041] Let's combine the following... Figure 1 This example illustrates the traditional process of a peripheral device accessing memory via the IOMMU.

[0042] like Figure 1 As shown, a CPU chip includes a root complex, which contains an IOMMU (Input / Output Management Unit) and a root port connected to the IOMMU. Peripherals connect to the IOMMU through this root port. Figure 1 In this context, peripherals include PCI bridges, GPUs, and network cards.

[0043] like Figure 1 As shown, the IOMMU includes IOTLB and DMAR (DMA remapping) units. The IOMMU uses the DMAR unit, IOTLB, and page tables to translate virtual memory addresses accessed by peripherals into physical memory addresses. In practical applications, virtual memory addresses can also be referred to as DMA addresses.

[0044] In practical applications, during the DMAR process of traversing the page table to find the physical memory address corresponding to the virtual memory address to be translated, two hardware units can be used: the page walk cache and the page walk engine. Simply put, the page walk engine completes the full traversal of the page table, while the page walk cache caches recently accessed page table levels. In practice, page tables can be multi-level.

[0045] In fact, such as Figure 1 As shown, page tables are stored in memory. Therefore, during page table traversal (i.e., page table lookup), if a multi-level page table is used, multiple memory access operations are required to find the physical memory address corresponding to the current virtual memory address.

[0046] Understandably, in Figure 1 The illustrated CPU chip may include one or more CPU cores. Based on the control of a certain CPU core, peripherals can initiate data read and write operations on memory.

[0047] Specifically, a peripheral device can send a DMA data read / write request to the IOMMU, which includes a virtual memory address. The IOMMU first checks the IOTLB to see if an address translation table entry corresponding to the virtual memory address exists. If it exists, the IOMMU performs the corresponding data read / write operation based on the physical memory address contained in the found address translation table entry, i.e., writing data to or reading data stored in that physical memory address. If it does not exist, the IOMMU performs a page table lookup to find the address translation table entry, and then performs the corresponding data read / write operation based on the physical memory address contained in the found address translation table entry. Simultaneously, the address translation table entry retrieved from the page table can be stored in the IOTLB. Thus, the next time a data read / write request corresponding to the virtual memory address is received, the translation from virtual memory address to physical memory address can be quickly completed based on the address translation table entry stored in the IOTLB.

[0048] Because the number of address translation entries stored in the page table is much greater than the number of address translation entries that can be stored in the IOTLB, and the page table is located in memory while the IOTLB is located locally in the IOMMU, the speed at which the IOMMU queries the IOTLB is much higher than the speed at which it queries the page table.

[0049] While IOTLB can improve address translation efficiency, like TLB, it is designed using an N-way set-associative approach, which is costly. Expanding the size of IOTLB not only increases hardware manufacturing costs but also increases the overall power consumption of the CPU chip. Furthermore, the lookup speed of IOTLB decreases as the IOTLB capacity increases.

[0050] In practical applications, an IOMMU can connect to one or more peripherals through a certain port. These peripherals consume the IOTLB in the IOMMU, which will quickly fill the IOTLB, resulting in an increase in the number of page table traversals and a decrease in I / O performance.

[0051] To improve I / O performance and reduce the number of page table traversals, this invention provides a new architecture for processing chips.

[0052] like Figure 2 As shown, this new processing chip includes at least one memory management unit, such as IOMMU1, IOMMU2, IOMMU3, and IOMMU4 illustrated in the figure. Each memory management unit includes a first address translation cache unit, such as L1-IOTLB1, L1-IOTLB2, L1-IOTLB3, and L1-IOTLB4 illustrated in the figure. In practical applications, the capacity of the first address translation cache units in these different memory management units can be the same. Of course, each memory management unit also includes an address translation processing unit, such as the DMAR unit illustrated in the figure.

[0053] In addition, the processing chip also includes a second address translation cache unit, such as the L2-IOTLB shown in the figure. The L2-IOTLB is connected to the different memory management units mentioned above via a high-speed bus.

[0054] Specifically, the second address translation cache unit can be connected to the address translation processing unit in different memory management units. In this way, the address translation processing unit in a memory management unit is connected to both the local first address translation cache unit and the second address translation cache unit.

[0055] In fact, the second address translation cache unit includes a control interface for connecting to the aforementioned high-speed bus. Similarly, each memory management unit also has a corresponding connection interface for connecting to this high-speed bus, specifically, as shown below. Figure 2 As shown, this connection interface can be connected to the address translation processing unit.

[0056] In practical applications, the capacity of the second address translation cache unit can be greater than that of the first address translation cache unit, and the sizes of both can be preset.

[0057] In some practical applications, in order not to significantly increase the overall hardware cost, the capacity of each first address translation cache unit can be reduced, and the saved resources can be used to construct a second address translation cache unit, as well as the aforementioned control interface and high-speed bus.

[0058] For example, suppose a traditional processing chip includes four IOMMUs, each containing a 4MB IOTLB. The total hardware overhead is 4MB * 4 = 16MB. Now suppose a new processing chip includes four IOMMUs, each containing a 3MB IOTLB. This effectively reduces the available space by 4MB to create the second IOTLB. In this case, the total hardware overhead is 3MB * 4 + 4MB = 16MB. It's clear that there isn't a significant increase in hardware cost. Of course, since the new processing chip also needs to construct the high-speed bus and access interface, the actual hardware cost will be slightly higher, but the overhead of this bus and access interface is not substantial.

[0059] When the capacity of the second address translation cache unit can be greater than that of the first address translation cache unit, the query speed of the second address translation cache unit is generally slower than that of the native first address translation cache unit (such as L1-IOTLB). This is because the query speed of IOTLB decreases as the capacity increases, but is still much higher than that of querying the page table.

[0060] In addition, it is understandable that Figure 2 The processing chip shown may also include, for example: Figure 1 The multiple CPU cores, root port, root node, and other components shown in the figure are not illustrated.

[0061] The following examples illustrate the application of... Figure 2 The memory access process of the processing chip is shown.

[0062] Figure 3 A flowchart of a memory access method provided in an embodiment of the present invention is provided. This method can be executed by a target memory management unit, which can be... Figure 2 The processing chip shown contains any memory management unit. For example... Figure 3 As shown, the method may include the following steps:

[0063] 301. The target memory management unit receives a data write request sent by the target external device. The data write request includes a virtual memory address. The target external device is connected to the target memory management unit.

[0064] 302. If the target memory management unit does not find an address translation table entry corresponding to the virtual memory address in the target first address translation cache unit, it checks whether the second address translation cache unit contains the address translation table entry. The address translation table entry contains the physical memory address corresponding to the virtual memory address. The target first address translation cache unit is the first address translation cache unit in the target memory management unit.

[0065] 303. If the target memory management unit does not find the address translation table entry in the second address translation cache unit, it queries the page table to determine the address translation table entry corresponding to the virtual memory address.

[0066] 304. The target memory management unit stores the address translation table entries retrieved from the page table into the target first address translation cache unit and the second address translation cache unit, respectively, and writes the data corresponding to the data write request into the physical memory address.

[0067] In this embodiment, it is assumed that the target external device (such as a network card, GPU, etc.) is connected to the target memory management unit contained in a processing chip through a root port. It is also assumed that the target memory management unit includes a target first address translation cache unit.

[0068] In practical applications, this processing chip contains one or more processor units, such as a CPU chip with multiple CPU cores. A target external device can trigger memory access under the control of a specific CPU core. For example, a process or virtual machine running on a CPU core sends a data write instruction to the target external device, carrying the virtual memory address. In fact, this data write instruction may also include the identifier of the process or virtual machine. Subsequently, the target external device sends a corresponding data write request, such as a DMA write request, to the target memory management unit, carrying the aforementioned virtual memory address and, possibly, the identifier of the process or virtual machine.

[0069] Next, the target memory management unit first checks the local target first address translation cache unit and second address translation cache unit sequentially to see if there is an address translation table entry corresponding to the virtual memory address. If not found, it can be considered that this is the first access to this virtual memory address, and then the page table corresponding to the process or virtual machine is traversed to obtain the address translation table entry. Then, the address translation table entry obtained from the page table is stored into the target first address translation cache unit and second address translation cache unit respectively, and according to the physical memory address corresponding to the virtual memory address contained in the obtained address translation table entry, the corresponding data write operation is performed, that is, the data contained in the above data write request is written into the storage space corresponding to the physical memory address in memory.

[0070] In practical applications, the memory management unit performs two main operations on the second address translation cache unit via the aforementioned high-speed bus: a write operation, i.e., a store operation (i.e., save new IOTLB entry), and a read operation, i.e., a load operation (i.e., lookup IOTLB). The bus has corresponding operation instructions for each operation: store instructions and load instructions. These instructions are used to complete the read and write operations on the second address translation cache unit.

[0071] Based on this, when the target memory management unit obtains the above address translation table entry, it stores the address translation table entry into its own target first address translation cache unit, and concurrently stores the address translation table entry into the second address translation cache unit through the store instruction of the bus.

[0072] Understandably, upon receiving the aforementioned data write request, if the target memory management unit finds an address translation table entry corresponding to the virtual memory address in the first target address translation cache unit, it will perform the corresponding data write operation on the memory based on the physical memory address contained in the address translation table entry found at that time. Alternatively, if the address translation table entry is not found in the first target address translation cache unit, but is found in the second address translation cache unit, the target memory will perform the corresponding data write operation on the memory based on the physical memory address contained in the address translation table entry found in the second address translation cache unit. In other words, the query order of the first target address translation cache unit takes precedence over that of the second address translation cache unit.

[0073] Therefore, assuming the target first address translation cache unit is small enough that the aforementioned address translation table entry was deleted from it at some point, this entry might still be stored in the second address translation cache unit. If the entry is found in the second address translation cache unit, then page table traversal is unnecessary. Thus, by increasing the number of address translation cache units available to each memory management unit—that is, by adding a large-capacity address translation cache unit shared by multiple memory management units in the processing chip—the number of page table traversals by the memory management units can be reduced, thereby improving I / O performance.

[0074] The above embodiments describe the process of writing data into memory. During this process, address translation table entries can be generated and stored in a first address translation cache unit local to the memory management unit and in a second address translation cache unit shared by different memory management units. The following will combine... Figure 4The illustrated embodiment describes the process of reading data from memory. For ease of description, it is assumed that the virtual memory address used in the reading process is the same as the virtual memory address used in the writing process described above.

[0075] Figure 4 A flowchart of a memory access method provided in an embodiment of the present invention is provided. This method can be executed by a target memory management unit, which can be... Figure 2 The processing chip shown contains any memory management unit. For example... Figure 4 As shown, the method may include the following steps:

[0076] 401. The target memory management unit receives a data read request sent by the target external device, and the data read request includes a virtual memory address.

[0077] 402. If the target memory management unit does not find an address translation table entry corresponding to the virtual memory address in the target first address translation cache unit, it checks whether the second address translation cache unit contains the address translation table entry, and also searches for the address translation table entry in the page table.

[0078] 403. If the target memory management unit obtains the address translation table entry from the second address translation cache unit, it stops querying the page table and reads data from the corresponding physical memory address based on the address translation table entry obtained from the second address translation cache unit.

[0079] In this embodiment, when the target memory management unit receives a data read request, such as a DMA read request, from a target external device, it first checks in the target first address translation cache unit whether there exists an address translation table entry corresponding to the virtual memory address carried therein. If it does not exist, for example, because the target first address translation cache unit may be full and the address translation table entry may be evicted, the target memory management unit can concurrently perform the following two actions:

[0080] First, traverse the page table;

[0081] Second, a load instruction is sent to the second address translation cache unit via the bus. This load instruction can carry the aforementioned virtual memory address and index information such as the process or virtual machine identifier, so that the second address translation cache unit can determine whether it contains an address translation table entry corresponding to the virtual memory address. If the second address translation cache unit returns the address translation table entry (for example, because the store operation mentioned above occurred recently), the target memory management unit cancels the page table traversal operation, directly stores the address translation table entry returned by the second address translation cache unit into the local target first address translation cache unit, and uses the physical memory address in the address translation table entry to perform a DMA read operation to read the data stored therein, and sends the read data to the target external device.

[0082] It should be noted that the concurrent execution of the two actions by the target memory management unit is only one optional implementation method, which can ensure better timeliness. Of course, in another optional implementation method, it is also possible to first check whether the second address translation cache unit contains the above address translation table entry, and if not, then perform the operation of traversing the page table.

[0083] As mentioned in the above embodiments, if the address translation cache unit is full, some previously stored address translation table entries will be deleted.

[0084] Regarding the aforementioned target first address translation cache unit, optionally, the target memory management unit can record the corresponding write time each time an address translation table entry is written to it. Thus, when the target first address translation cache unit is full, the oldest written address translation table entry can be deleted based on the write time, allowing a new address translation table entry to be written. Alternatively, optionally, the target memory management unit can also record the latest query time and the number of queries for each address translation table entry already written to the target first address translation cache unit, using the latest query time and the number of queries as usage frequency parameters for the corresponding address translation table entry. When the target first address translation cache unit is full, address translation table entries with fewer queries can be deleted first, or address translation table entries with older latest query times can be deleted first. Of course, in this embodiment, the decision can also be made in conjunction with the write time of the address translation table entries.

[0085] If it is ultimately determined that an address translation table entry needs to be deleted from the target first address translation cache unit, the target memory management unit may optionally send a store instruction to the second address translation cache unit to instruct the storage of the address translation table entry, so that the second address translation cache unit stores the address translation table entry when it is determined that the address translation table entry is not stored.

[0086] In practical applications, since the second address translation cache unit is shared by multiple memory management units and can be used to store address translation table entries transmitted from each memory management unit, there may be situations where an address translation table entry has been removed from the second address translation cache unit, but is still stored in a first address translation cache unit. Therefore, when the target memory management unit determines that an address translation table entry needs to be deleted from the target first address translation cache unit, it sends a store instruction to the second address translation cache unit, instructing it to store the address translation table entry, in order to delay the retention time of the address translation table entry in the address translation cache unit as much as possible.

[0087] After receiving the above-mentioned store instruction, the second address translation cache unit first determines whether the address translation table entry is already stored locally. If it is already stored, the store instruction is ignored; if it is not stored, the address translation table entry is stored.

[0088] In summary, in this embodiment of the invention, by expanding the number of address translation cache units available when external devices perform address translation through the memory management unit, the number of page table traversals can be reduced, thereby improving IO performance.

[0089] The following section describes the specific implementation process of the memory access method provided by the embodiments of the present invention in some application scenarios, using several specific application scenarios as examples.

[0090] Scenario 1: In a high-speed cloud computing scenario, suppose a server contains components from... Figure 5 The image shows a CPU chip that is installed on the server's motherboard. This CPU chip contains multiple cores, such as... Figure 5 The diagram illustrates N cores. Additionally, it assumes that the CPU chip includes multiple memory management units, such as... Figure 5 The diagram illustrates two of them: IOMMU1 and IOMMU2. Each memory management unit includes a first address translation cache unit (IOTLB1, IOMMU2) and an address translation processing unit (DMAR). Furthermore, in Figure 5 In this example, IOMMU1 is connected to root port 1, and IOMMU2 is connected to root port 2.

[0091] Assuming a physical network card is inserted into the motherboard of this server, and virtualization technology is used to generate... Figure 5 The diagram illustrates N virtual network adapters, and these N virtual network adapters are assigned one-to-one to N virtual machines (VMs). Figure 5(Not shown in the diagram) These N VMs run one-to-one on the N CPU cores shown in the diagram. Furthermore, all virtual network cards are connected to the same root port 1, thus a single IOMMU1 manages the I / O traffic of the N VMs, and all DMA read / write requests from the virtual network cards consume IOTLB1 in IOMMU1. Meanwhile, other IOMMUs in the CPU chip, such as IOMMU2, are idle because their corresponding root port 2 has no peripherals connected. Therefore, the overall IOTLB utilization of the machine is very low. For example, assuming there are a total of 20 IOMMUs, if the IOTLBs of the remaining 19 IOMMUs (excluding IOMMU1) are all idle, the overall IOTLB utilization of the machine is only about 5%, resulting in a significant waste of resources.

[0092] As the core density of CPU chips continues to increase, the computational load on a single CPU chip is also rising. With multiple virtual network interface cards (NICs) still connected to a single root port, the IOTLB capacity of a single IOMMU cannot meet the ever-increasing computational demands, and the increase in IOTLB capacity within each IOMMU cannot keep pace with the increase in core density. When IOTLBs are full, a large number of address translation table entries will be removed, resulting in additional page table traversals and negatively impacting I / O performance.

[0093] In this situation, by adopting the solution provided in the embodiments of the present invention, an additional large-capacity IOTLB is set up in the CPU chip for all IOMMUs to share, and based on the memory access method provided in the aforementioned embodiments, the number of page table traversals can be reduced, thereby improving I / O performance.

[0094] In this embodiment, we will refer to the additional IOTLB shared by multiple IOMMUs as L2-IOTLB. This will be used when running on... Figure 5 Taking VM1 on CPU core 1 as an example, assuming VM1 uses virtual network card 1, the data writing process of VM1 can be as follows:

[0095] VM1 triggers a data write request to virtual network interface card 1, which carries the virtual memory address, the VM1 identifier, and the data to be written to memory;

[0096] Virtual network card 1 sends a DMA write request to the connected IOMMU1, which carries the aforementioned virtual memory address and the identifier of VM1;

[0097] IOMMU1 checks in the local IOTLB1 whether there is an address translation table entry 1 corresponding to the virtual memory address and VM1 identifier. If not, it checks in the L2-IOTLB whether there is an address translation table entry 1 corresponding to the virtual memory address and VM1 identifier. If not, it traverses the page table 1 corresponding to the VM1 identifier to find the address translation table entry 1 corresponding to the virtual memory address.

[0098] After obtaining address translation table entry 1 from page table 1, IOMMU1 stores address translation table entry 1 into IOTLB1 and L2-IOTLB respectively, and writes the data to be written into the corresponding physical memory address based on the physical memory address in address translation table entry 1.

[0099] In practice, each VM using different virtual network adapters can trigger the above data write process to complete the data write operation to memory using the same IOMMU1. Since all VMs share a single IOMMU1, assuming IOTLB1 can store 1000 address translation table entries, it will quickly become full. However, because the L2-IOTLB has a larger capacity, for example, it can store 4000 address translation table entries, and address translation table entries are also stored in the L2-IOTLB when they are stored in IOTLB1, the same address translation table entry may remain in the L2-IOTLB for a longer time. Therefore, when this address translation table entry is needed for address translation processing later, it can still be found in the L2-IOTLB, requiring a page table traversal, thus reducing the number of page table traversals.

[0100] Understandably, in practical applications, different memory regions may be allocated to different virtual network cards. Therefore, the memory access regions of VMs using different virtual network cards are different, that is, the range of physical memory addresses corresponding to different VMs are different. It is necessary to maintain different page tables for different VMs. The difference in these page tables is mainly reflected in the difference in physical memory addresses, while the virtual memory addresses can be the same.

[0101] Therefore, the VM identifier carried in the above data read / write requests is used to distinguish the page tables corresponding to different VMs and the address translation entries in the page tables. For example, the same virtual memory address IOVAi corresponds to the physical memory address HPAi1 in page table 1 corresponding to VM1, and to the physical memory address HPAi2 in page table 2 corresponding to VM2.

[0102] In the above scenario, the data reading process can be performed by referring to the data reading process described in the other embodiments above, and will not be elaborated here.

[0103] Scenario 2: Heterogeneous computing scenario.

[0104] In heterogeneous computing scenarios, other types of processors are mainly used to assist the CPU core in computation. For example, in image or video data processing tasks, the CPU core can control the network card to write the received image or video data into memory and inform the GPU of the virtual memory address used when writing to memory. Based on this virtual memory address, the GPU ultimately reads the corresponding image or video data from memory for computation and processing, thus achieving the purpose of the GPU assisting the CPU core in computation.

[0105] For ease of understanding, combined with Figure 6 This example illustrates a heterogeneous computing scenario.

[0106] exist Figure 6 In this example, assume that a CPU core is running VM1. VM1 can use two peripherals: a network card and a GPU. Furthermore, assume that the network card is connected to IOMMU1 via root port 1, and the GPU is connected to IOMMU2 via root port 2. Assume that IOMMU1 includes IOTLB1, and IOMMU2 includes IOTLB2. VM1 has a corresponding page table 1.

[0107] Without using the solution provided in the embodiments of the present invention, the data read / write process is as follows:

[0108] VM1 triggers a data write request to the network card, which carries the virtual memory address, VM1's identifier, and the data to be written to memory;

[0109] The network card sends a DMA write request to the connected IOMMU1, which carries the aforementioned virtual memory address and the identifier of VM1;

[0110] IOMMU1 checks in the local IOTLB1 whether there is an address translation table entry 1 corresponding to the virtual memory address and VM1 identifier. If not, it traverses the page table 1 corresponding to the VM1 identifier to find the address translation table entry 1 corresponding to the virtual memory address, stores the obtained address translation table entry 1 into IOTLB1, and writes the data to be written to the corresponding physical memory address based on the physical memory address in address translation table entry 1.

[0111] IOMMU1 triggers a data read request to the GPU, which carries the aforementioned virtual memory address and the identifier of VM1;

[0112] The GPU checks if there is an address translation table entry 1 corresponding to the virtual memory address and VM1 identifier in the local IOTLB2. If not, it traverses the page table 1 corresponding to the VM1 identifier to find the address translation table entry 1 corresponding to the virtual memory address. Based on the physical memory address in the found address translation table entry 1, it reads data from the physical memory address for calculation and processing.

[0113] Therefore, it can be seen that different peripherals operating under the same VM have a certain probability of accessing the same address. Since the IOMMUs connected to different peripherals are physically separated, they can only use their own internal IOTLBs. When they all need to perform page table traversal on the same virtual memory address, the repeated page table traversal on the same virtual memory address will lead to a decrease in I / O performance.

[0114] However, if the solution provided in the embodiments of the present invention is adopted in this scenario, an additional large-capacity IOTLB is set up in the CPU chip for all IOMMUs to share, and the memory access method provided in the aforementioned embodiments is used to reduce the number of page table traversals and improve IO performance.

[0115] In this embodiment, the additional IOTLB shared by multiple IOMMUs is referred to as L2-IOTLB. Figure 6 In the illustrated heterogeneous computing scenario, the data read / write process is as follows:

[0116] VM1 triggers a data write request to the network card, which carries the virtual memory address, VM1's identifier, and the data to be written to memory;

[0117] The network card sends a DMA write request to the connected IOMMU1, which carries the aforementioned virtual memory address and the identifier of VM1;

[0118] IOMMU1 checks if there is an address translation table entry 1 corresponding to the virtual memory address and VM1 identifier in the local IOTLB1. If not, it checks if there is an address translation table entry 1 corresponding to the virtual memory address and VM1 identifier in the L2-IOTLB. If not, it traverses the page table 1 corresponding to the VM1 identifier to find the address translation table entry 1 corresponding to the virtual memory address. The obtained address translation table entry 1 is stored in IOTLB1, and the data to be written is written to the corresponding physical memory address based on the physical memory address in the address translation table entry 1.

[0119] IOMMU1 triggers a data read request to the GPU, which carries the aforementioned virtual memory address and the identifier of VM1;

[0120] The GPU checks if an address translation table entry 1 corresponding to the virtual memory address and VM1 identifier exists in its local IOTLB2. If not, it concurrently checks if an address translation table entry 1 corresponding to the virtual memory address and VM1 identifier exists in the L2-IOTLB, and traverses the page table 1 corresponding to the VM1 identifier. If an address translation table entry 1 is found in the L2-IOTLB, it stops traversing the page table 1, stores the address translation table entry 1 in IOTLB1, and reads data from the physical memory address based on the physical memory address found in the address translation table entry 1 for computation and processing.

[0121] Therefore, if the L2-IOTLB contains the address translation table entry 1 mentioned above, the traversal of page table 1 will terminate. Concurrently querying the L2-IOTLB and page table 1 will not incur additional time consumption, thus ensuring IO performance and reducing the number of times the page table is repeatedly traversed for the same virtual memory address.

[0122] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 7 As shown, the electronic device includes at least one processing chip (such as processing chip 1 and processing chip 2 shown in the figure), memory, and external devices. Each processing chip may include at least one memory management unit, and each memory management unit includes a first address translation cache unit and an address translation processing unit. Additionally, each processing chip includes a second address translation cache unit shared by all its memory management units, connected to each memory management unit via a bus.

[0123] Different external devices (e.g.) Figure 7 External devices 1 and 2, as shown in the diagram, can be connected to different memory management units via the root port.

[0124] Each memory management unit includes an address translation processing unit, which can be configured to execute the memory access methods provided in the foregoing embodiments.

[0125] In addition, embodiments of the present invention provide a non-transitory machine-readable storage medium storing executable code, wherein when the executable code is executed by a memory management unit in an electronic device, the memory management unit can at least implement the memory access method provided in the foregoing embodiments.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A memory access method, characterized in that, The method is applied to a target memory management unit included in a processing chip, wherein the target memory management unit is any one of at least one memory management unit included in the processing chip, and each memory management unit includes a first address translation cache unit; the method includes: Receive a data write request sent by a target external device, the data write request including a virtual memory address, the target external device being connected to the target memory management unit; If no address translation table entry corresponding to the virtual memory address is found in the first target address translation cache unit, then the second address translation cache unit is queried to see if the address translation table entry is contained; wherein, the address translation table entry contains the physical memory address corresponding to the virtual memory address, the second address translation cache unit is connected to the at least one memory management unit, and the first target address translation cache unit is the first address translation cache unit in the target memory management unit; If the address translation table entry is not found in the second address translation cache unit, the page table is queried to determine the address translation table entry corresponding to the virtual memory address; The address translation table entries retrieved from the page table are respectively stored in the target first address translation cache unit and the second address translation cache unit; Write the data corresponding to the data write request to the physical memory address.

2. The method according to claim 1, characterized in that, The capacity of the second address translation cache unit is greater than that of the first address translation cache unit.

3. The method according to claim 1, characterized in that, The method further includes: If the address translation table entry is found in the target first address translation cache unit, then the memory is written with data corresponding to the data write request according to the physical memory address contained in the address translation table entry; or, If the address translation table entry is found in the second address translation cache unit, then the memory is written with data corresponding to the data write request according to the physical memory address contained in the address translation table entry. The query order of the first target address translation cache unit is prioritized over that of the second address translation cache unit.

4. The method according to claim 1, characterized in that, The method further includes: Receive a data read request sent by the target external device, wherein the data read request includes the virtual memory address; If the address translation table entry is not found in the first target address translation cache unit, then the second address translation cache unit is queried to see if the address translation table entry is contained, and the address translation table entry is queried in the page table; If the address translation table entry is obtained from the second address translation cache unit, then the query of the page table is stopped; Data is read from the physical memory address based on the address translation table entry retrieved.

5. The method according to claim 1, characterized in that, The method further includes: If it is determined that the address translation table entry will be deleted from the target first address translation cache unit, a store instruction instructing the second address translation cache unit to store the address translation table entry is sent, so that the second address translation cache unit stores the address translation table entry when it is determined that the address translation table entry is not stored.

6. A memory management unit, characterized in that, include: A first address translation cache unit and an address translation processing unit; wherein, the address translation processing unit is connected to a second address cache unit, the second address cache unit is connected to different address translation processing units located within the same processing chip, and the different address translation processing units are located in different memory management units within the processing chip; The address translation processing unit is used to execute the memory access method as described in any one of claims 1 to 5.

7. The memory management unit according to claim 6, characterized in that, The memory management unit includes an IOMMU, the first address translation cache unit includes a first IOTLB, and the second address translation cache unit includes a second IOTLB.

8. A processing chip, characterized in that, include: At least one memory management unit as described in claim 6, and a second address translation cache unit; The second address cache unit is connected to each memory management unit.

9. An electronic device, characterized in that, include: At least one processing chip, memory, and external device as described in claim 8.

10. A non-transitory machine-readable storage medium, characterized in that, The non-transitory machine-readable storage medium stores executable code, which, when executed by a memory management unit in the electronic device as claimed in claim 9, causes the memory management unit to perform the memory access method as claimed in any one of claims 1 to 5.

Images