Memory access method and apparatus, storage medium, and program product

Abstract

Embodiments of the present application provide a memory access method and device, a storage medium and a program product, and relate to the technical field of computers. The mapping state between target data and memory is represented by more abundant state information, the page fault problem is avoided, the memory access process is simplified, and the memory access overhead is reduced. The method comprises the following steps: a heterogeneous accelerator receives a memory access request sent by a virtual instance through a pass-through channel; in response to the memory access request, target state information corresponding to a target address is obtained; the memory access request carries the target address and is used to request access to a target storage space in the memory corresponding to the target address; in the case that the target state information indicates a pre-unmapping state, a first instruction is sent to a computing device; the pre-unmapping state indicates that the computing device is about to cancel a target mapping relationship that has been established between the target address and the target storage space; the first instruction is used to instruct that the computing device is not allowed to cancel the target mapping relationship; and the target storage space is accessed through the target address.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to memory access methods, apparatus, storage media and program products. Background Technology

[0002] Currently, virtualization technology is widely used in cloud computing scenarios, such as deploying multiple virtual machines (VMs) on the same physical server (host) to provide corresponding services. Resource scheduling technology is used to dynamically adjust resource allocation to meet the needs of each VM. For example, physical memory no longer needed by one VM can be allocated to another. To improve performance, VMs can use hardware pass-through to more directly and efficiently control and use dedicated computing units (including data processing units) installed on the physical server. For example, to improve the efficiency of VMs in network communication, VMs can use smart network interface cards (NICs) through hardware pass-through.

[0003] In this situation, the virtual machine management unit cannot detect the smart network card's access to the host machine's memory, which leads to a memory page fault and causes memory access failure. Summary of the Invention

[0004] This application provides a memory access method, apparatus, storage medium, and program product. It characterizes the mapping state between the data to be accessed by the heterogeneous accelerator card and the physical memory of the computing device through a mapping state, a pre-demapping state, and a demapping state, thereby avoiding page faults. Furthermore, before the heterogeneous accelerator card accesses memory, there is no need for additional interaction between it and the computing device, simplifying the memory access process and reducing memory access overhead.

[0005] Firstly, this application provides a memory access method applied to a heterogeneous accelerator card, which is inserted into a computing device. The computing device includes memory and runs a virtual instance. A pass-through channel is established between the heterogeneous accelerator card and the virtual instance. Specifically, the method includes: receiving a memory access request sent by the virtual instance through the pass-through channel; responding to the memory access request by obtaining target status information corresponding to a target address; wherein the memory access request carries a target address for requesting access to a target storage space in the memory corresponding to the target address; when the target status information indicates a pre-demapping state, sending a first instruction to the computing device; the pre-demapping state indicates that the computing device is about to release the established target mapping relationship between the target address and the target storage space; the first instruction is used to indicate that the computing device is not allowed to release the target mapping relationship; and accessing the target storage space through the target address.

[0006] Understandably, the pre-demapping state is a transitional state between the mapping state and the demapping state. If the heterogeneous accelerator card finds that the target address's state information is in the pre-demapping state before accessing memory, it can send a first instruction to prevent the computing device from releasing the target mapping relationship, thus avoiding page faults and improving system reliability. Furthermore, the heterogeneous accelerator card eliminates the need for additional interaction with the computing device, simplifying the memory access process and reducing memory access overhead.

[0007] In one possible implementation, in response to a memory access request, the heterogeneous accelerator card obtains the target status information corresponding to the target address, including: in response to the memory access request, the heterogeneous accelerator card queries a first mapping relationship set to obtain the target status information corresponding to the target address; wherein, the first mapping relationship set is stored in the heterogeneous accelerator card, and the first mapping relationship set includes at least one address and its corresponding status information, and the at least one address includes the physical address of the target storage space in memory.

[0008] Understandably, by querying the first mapping set, the heterogeneous accelerator card can accurately determine the target status information corresponding to the target address, and then make the correct subsequent processing operations, avoiding the chaos caused by the erroneous status, and ensuring the correctness and reliability of memory access.

[0009] In one possible implementation, in the first mapping relationship set, state information is carried through at least one field, indicating a mapping state, a demapping state, or a pre-demapping state; wherein, the mapping state indicates that the computing device maintains an established mapping relationship between at least one address and a storage space in memory; the demapping state indicates that the computing device has demapped the relationship.

[0010] Understandably, by using three different states, it is possible to avoid switching only between the mapping state and the demapping state (two completely opposite states), and the page fault problem is effectively avoided by using the pre-demapping state.

[0011] In one possible implementation, the status information is the frequency of accessing the corresponding storage space in memory through the address in the first mapping relationship set; when the frequency meets the target range, the status information indicates the pre-demapping state; when the frequency is higher than the target range, the status information indicates the mapping state; when the frequency is lower than the target range, the status information indicates the demapping state.

[0012] It is understandable that a higher access frequency indicates higher data popularity, so it should reside in memory for a longer period of time. By comparing the frequency with the target range, it is possible to efficiently and accurately distinguish between the three different state information, effectively extend the residence time of hot data, reduce the inefficient use of storage space by cold data, and help improve resource utilization.

[0013] In one possible implementation, after accessing the target storage space via the target address, the method further includes: if the number of times the target storage space is accessed via the target address within a first predetermined time period is less than a first threshold, modifying the target status information; wherein the modified target status information indicates a demapping state; the demapping state indicates that the computing device has released the target mapping relationship; or...

[0014] If the number of times the target storage space is accessed through the target address exceeds a second threshold within a second set time period, the target status information is modified; wherein, the modified target status information indicates a mapping state; the mapping state indicates that the computing device maintains the target mapping relationship.

[0015] Understandably, heterogeneous accelerator cards can efficiently and accurately adjust target status information by calculating the number of times (or access frequency) the target storage space is accessed through the target address within a set time period, thereby avoiding page faults.

[0016] In one possible implementation, after modifying the target state information, the method further includes synchronizing the modified target state information to the computing device.

[0017] Understandably, timely synchronization of heterogeneous accelerator cards with computing devices after modifying status information helps maintain consistency between the two devices, thereby ensuring the stability of memory access and preventing the erroneous removal of mapping relationships involving hot data or the long-term retention of mapping relationships for cold data.

[0018] In one possible implementation, before obtaining the target state information corresponding to the target address in response to a memory access request, the method further includes: receiving a synchronization instruction sent by a computing device, the synchronization instruction being used to instruct the state indicated by the target state information corresponding to the target address to be synchronized to a pre-demapping state; and synchronizing the state indicated by the target state information to a pre-demapping state.

[0019] Understandably, heterogeneous accelerator cards can receive synchronization commands from computing devices and update the status information corresponding to the target address in a timely manner, maintaining the consistency of information between the two devices and thus avoiding page faults.

[0020] In one possible implementation, the target status information corresponding to the target address is related to the frequency of accessing the target storage space.

[0021] It is understandable that the higher the access frequency, the higher the popularity of the storage space (data). Therefore, the relevant mapping relationship should be preserved. Thus, frequency can effectively and accurately distinguish different state information, ensuring the accuracy and reliability of memory access.

[0022] In one possible implementation, if the pre-demapping state specifically indicates that the computing device is about to remove data from the target storage space, then the first instruction is specifically used to indicate that the computing device is not allowed to remove data from the target storage space.

[0023] Understandably, when the pre-demapping state specifically indicates that the computing device is about to remove data from the target storage space, the first instruction issued by the heterogeneous accelerator card should also be more specific, indicating that the computing device is not allowed to remove the data to avoid memory access anomalies.

[0024] Secondly, this application provides another memory access method, which is applied to a computing device, the computing device including memory, and a heterogeneous accelerator card inserted in the computing device, a virtual instance running on the computing device, and a pass-through channel established between the heterogeneous accelerator card and the virtual instance; the method includes: receiving a first instruction sent by the heterogeneous accelerator card, the first instruction being used to indicate that the target mapping relationship between the target address and the target state information is not allowed to be removed, the state indicated by the target address is a pre-demapping state, the pre-demapping state indicating that the computing device is about to remove the established target mapping relationship between the target address and the target storage space; in response to the first instruction, the target mapping relationship is not removed temporarily.

[0025] Understandably, the computing device, under the instruction sent by the heterogeneous accelerator card, did not remove the target mapping relationship, ensuring that the heterogeneous accelerator card could still access data normally and avoiding page faults.

[0026] In one possible implementation, the computing device stores a second mapping set, in which state information is carried by at least one field, indicating a mapping state, a demapping state, or a pre-demapping state; wherein, the mapping state indicates that the computing device maintains an established mapping relationship between at least one address and a storage space in memory; the demapping state indicates that the computing device has demapped the relationship.

[0027] Understandably, the state information corresponding to each address in the second mapping set can include three different states, avoiding the need to switch only between the mapping state and the demapping state (two completely opposite states). The pre-demapping state fills the technical gap between the two, effectively preventing the occurrence of page faults.

[0028] In one possible implementation, the status information is the frequency of the heterogeneous accelerator card accessing the corresponding storage space in memory; when the frequency meets the target range, the status information indicates the pre-demapping state; when the frequency is higher than the target range, the status information indicates the mapping state; when the frequency is lower than the target range, the status information indicates the demapping state.

[0029] It is understandable that a higher access frequency indicates higher data popularity, and therefore the data should reside in the memory of the computing device for a longer period of time. By comparing the frequency with the target range, the computing device can efficiently and accurately distinguish between three different state information, effectively extending the residence time of hot data, reducing the inefficient use of storage space by cold data, and helping to improve resource utilization.

[0030] In one possible implementation, the method further includes: the computing device counting the frequency of heterogeneous accelerator cards accessing the target storage space; if the frequency is lower than a first threshold, modifying the target status information; wherein the modified target status information indicates a demapping state; the demapping state indicates that the computing device has released the target mapping relationship; or, if the frequency is higher than a second threshold, modifying the target status information; wherein the modified target status information indicates a mapping state; the mapping state indicates that the computing device maintains the target mapping relationship.

[0031] It is understandable that by statistically analyzing the frequency of access to the target storage space by the heterogeneous accelerator card (the aforementioned target storage space can be any storage space allocated by the memory in the computing device to the heterogeneous accelerator card), the computing device can efficiently and accurately adjust the target state information and comply with the corresponding operating rules, thereby avoiding page faults. The computing device can also reallocate the portion of memory corresponding to the demapped state in a timely manner, thereby improving resource utilization.

[0032] In one possible implementation, after updating the target state information according to frequency, the method further includes: the computing device synchronizing the modified target state information to the heterogeneous accelerator card.

[0033] It is understandable that by promptly synchronizing the target state information modified by the computing device with the heterogeneous accelerator card, the consistency of information between the two devices can be maintained, thereby avoiding problems that may be caused by information inconsistency and improving system stability.

[0034] In one possible implementation, the method further includes: receiving a synchronization instruction sent by a heterogeneous accelerator card; the synchronization instruction is used to instruct the state indicated by the target state information corresponding to the target address to be synchronized to the pre-demapping state; and synchronizing the state indicated by the target state information to the pre-demapping state.

[0035] Understandably, by receiving synchronization commands from heterogeneous accelerator cards, computing devices can update the status information corresponding to the target address in a timely manner, maintain the consistency of information between the two devices, and thus avoid page faults.

[0036] Thirdly, this application provides a memory access device for executing any of the memory access methods provided in the first aspect above.

[0037] In one possible implementation, this application can divide the memory access device into functional modules according to the method provided in the first aspect above. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one module. For example, this application can divide the memory access device into an acquisition module, a sending module, and an access module according to function. The descriptions of the possible technical solutions and beneficial effects performed by the above-described functional modules can refer to the technical solutions provided in the first aspect above or its corresponding possible implementations, and will not be repeated here.

[0038] Fourthly, this application provides another memory access device for performing any of the memory access methods provided in the second aspect above.

[0039] In one possible implementation, this application can divide the memory access device into functional modules according to the method provided in the first aspect above. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one module. For example, this application can divide the memory access device into a receiving module and a releasing module, etc., according to function. The descriptions of the possible technical solutions and beneficial effects performed by the above-described functional modules can refer to the technical solutions provided in the first aspect above or its corresponding possible implementations, and will not be repeated here.

[0040] Fifthly, embodiments of this application provide a computing device comprising a processor and a memory, the processor being coupled to the memory; the memory is used to store computer instructions, which are loaded and executed by the processor to enable the computing device to implement the memory access methods provided in the first or second aspect and various alternative implementations described above.

[0041] In a sixth aspect, embodiments of this application provide a computing device cluster, the computing device cluster including at least one computing device, each computing device including a processor and a memory, the processor being coupled to the memory; the processor of the at least one computing device is used to execute computer instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the memory access methods provided in the first or second aspect and various optional implementations described above.

[0042] In a seventh aspect, embodiments of this application provide a computer-readable storage medium storing at least one computer program instruction, which is loaded and executed by a heterogeneous accelerator card or computing device to implement the memory access method provided in the first or second aspect and various optional implementations described above.

[0043] Eighthly, embodiments of this application provide a computer program product including computer instructions stored in a computer-readable storage medium. A computing unit of a heterogeneous accelerator card or computing device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the heterogeneous accelerator card or computing device to perform the memory access methods provided in the first or second aspect and various optional implementations described above.

[0044] For a detailed description of aspects three through eight and their various implementations in this application, please refer to the detailed description in aspect one or two and their various implementations; and for a detailed analysis of the beneficial effects of aspects three through eight and their various implementations in aspect one or two and their various implementations, please refer to the beneficial effect analysis in aspect one or two and their various implementations, which will not be repeated here.

[0045] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of a traditional virtualization technology;

[0047] Figure 2 This is a schematic diagram of a virtualized heterogeneous architecture;

[0048] Figure 3 This is a diagram illustrating a page fault.

[0049] Figure 4 This is a schematic diagram of an interaction process related to a traditional technology.

[0050] Figure 5 This application provides an embodiment of an interactive diagram illustrating the access of a heterogeneous accelerator card to the memory of a computing device.

[0051] Figure 6 This application provides a schematic diagram of the architecture of a cloud computing system.

[0052] Figure 7 This is a schematic diagram of the structure of a heterogeneous accelerator card provided in an embodiment of this application;

[0053] Figure 8 An interactive flowchart of a memory access method provided in an embodiment of this application;

[0054] Figure 9 A schematic diagram comparing the state information before and after the improvement provided in this application embodiment;

[0055] Figure 10 A schematic diagram illustrating a change in status information provided in an embodiment of this application;

[0056] Figure 11 This is a schematic diagram of the structure of a memory access device provided in an embodiment of this application;

[0057] Figure 12 This is a schematic diagram of another memory access device provided in an embodiment of this application;

[0058] Figure 13 A schematic diagram of a computing device provided in an embodiment of this application;

[0059] Figure 14 A schematic diagram of a computing device cluster provided in an embodiment of this application;

[0060] Figure 15 This is a schematic diagram illustrating a connection method between computing device clusters provided in an embodiment of this application. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0062] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0063] Furthermore, in the description of the embodiments of this application, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0064] Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" are not necessarily different. Meanwhile, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is being used as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present related concepts in a concrete manner for ease of understanding.

[0065] First, the application scenarios of the embodiments of this application will be introduced by way of example.

[0066] In traditional cloud computing scenarios, virtualization technology, as one of the core technologies, often abstracts physical resources into virtual resources through a hypervisor. Traditional virtualization typically employs... Figure 1 The homogeneous architecture shown Figure 1 This is a schematic diagram of a traditional virtualization technology, which provides a standardized virtual hardware interface on all virtual machines, and the hypervisor centrally schedules the resources of the physical computing devices, such as the central processing unit (CPU) and memory.

[0067] in, Figure 1 The operating system (OS) running on the physical computing device 1 shown can be called the host OS, and the operating system running on the virtual machine can be called the guest OS. Figure 1 The guest OS running on virtual machine 1 and the guest OS running on virtual machine 2 can be the same type of operating system or different types of operating systems.

[0068] To improve the utilization of computing resources, various techniques can be used for optimization. For example, memory optimization techniques such as memory overcommitment, memory late allocation, and memory swap can enable the running of one (or more) virtual machines with a total memory requirement of more than 24 GB on a physical computing device with only 24 gigabytes (GB) of physical memory.

[0069] However, Figure 1 In the architecture shown, when the virtual machine needs to operate the real physical hardware, it needs the hypervisor to act as an intermediary. This intermediary process requires information exchange and also consumes input / output (I / O) bandwidth, resulting in low efficiency, high latency, and performance loss.

[0070] With the explosive growth in demand for heterogeneous computing, traditional cloud service architectures are also changing: by introducing heterogeneous accelerator cards other than traditional CPUs, such as graphics processing units (GPUs), neural network processing units (NPUs), data processing units (DPUs), application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs), hybrid computing power pools are built to meet the differentiated computing needs of big data, artificial intelligence, and high-performance networks.

[0071] Hardware pass-through technology is one of the key technologies in this process. Based on mechanisms such as single root I / O virtualization (SR-IOV) and virtual function I / O (VFIO) in the high-speed serial computer expansion bus standard (PCIe), virtual machines can directly penetrate the hypervisor and establish a direct connection channel with dedicated acceleration units, thereby directly accessing the dedicated acceleration units, for example... Figure 2 As shown, Figure 2 This is a schematic diagram of a virtualized heterogeneous architecture. Figure 2 The virtual machines 3 and 4 shown run on physical computing device 2. The smart network interface card (SmartNIC) is mounted to virtual machine 3 via hardware pass-through (meaning a direct connection is established between virtual machine 3 and the smart NIC). This allows virtual machine 3 to directly control the smart NIC with near-physical machine performance, significantly improving performance and efficiency in performing network communication-related tasks.

[0072] However, this method also brings some problems. Because it penetrates the hypervisor, it... Figure 2 The hypervisor in the system has difficulty monitoring communication behavior occurring in the direct connection channel, and therefore cannot notify the physical computing device 2 (host OS) in a timely manner.

[0073] Consequently, in heterogeneous virtualization scenarios, when dynamically scheduling memory, the host OS may not be aware of memory access requests sent by Virtual Machine 3 to the smart network card via the direct connection. This can lead to page faults (data may be swapped to disk instead of being in physical memory), causing memory access failures. Furthermore, in heterogeneous virtualization scenarios, pass-through devices often use direct memory access (DMA) to access physical memory. This method does not require the host CPU's involvement (or is CPU-insensitive), further exacerbating the occurrence of page faults.

[0074] See details Figure 3 , Figure 3 This diagram illustrates a page fault. In traditional solutions (including the three described below), the mapping set (which can refer to the page table) has only two states for indicating whether a page has been loaded into physical memory: mapped (loaded) and demapped (not loaded). When the heterogeneous accelerator card performs three DMA transactions, transactions 1 and 2 complete normally, but transaction 3 is not finished when the page state suddenly jumps from the mapped state to the demapped state. The relevant data may be moved, causing a serious access failure (page fault).

[0075] To address the page fault issue encountered when passing devices access host memory in heterogeneous virtualization scenarios, and in other words, to simultaneously leverage the high performance benefits brought by memory optimization technologies (such as super-resolution), hardware passthrough, and DMA, the industry currently offers the following three immature solutions.

[0076] Solution 1: Monitor the DMA buffer within the virtual machine to avoid DMA failures.

[0077] This method requires adding a DMA monitoring module to the virtual machine. When the virtual machine allocates or reclaims DMA-related memory, it actively sends the relevant information to the hypervisor. Upon receiving the notification, the hypervisor locks the corresponding memory to avoid page faults. However, the DMA monitoring module raises serious security concerns.

[0078] Option 2: Use a CPU that supports I / O page fault (IOPF)

[0079] This approach requires specific hardware, making it difficult to implement with the vast amount of existing computing devices. Furthermore, regarding the specific technical details of this approach...

[0080] 1) If a page fault occurs during DMA, the I / O memory management unit (IOMMU) in the computing device will pause the DMA operation, record the relevant error information in the event queue, and notify the CPU (host OS) via an interrupt.

[0081] 2) The memory management subsystem in the host OS fills in the page fault and sends a recovery command to the IOMMU.

[0082] 3) The IOMMU notifies the device to retry the DMA operation.

[0083] It is clear that this approach does not prevent page faults from occurring; rather, it serves as a remedial measure after a page fault has occurred, involving an interrupt to the CPU (which incurs significant performance overhead) and page fault recovery. Predictably, with frequent page faults, this approach will severely consume CPU resources to handle these exceptions, thus hindering the efficient utilization of computational performance.

[0084] Option 3: Support partial address translation on the device side.

[0085] In this scheme, the heterogeneous accelerator card and the computing device can each maintain a portion of the address translation results and keep them synchronized. However, due to the unavoidable latency in the synchronization process, in addition to synchronization, the following additional interaction process is required (see...). Figure 4 This is to avoid page missing issues.

[0086] Figure 4 This is a schematic diagram of an interaction process related to a traditional technology. Before the device initiates a DMA operation, if it finds that the data has not been loaded into memory, it needs to send a page request to ensure that the page has been reloaded into physical memory (corresponding to...). Figure 4 Then proceed with steps S1-S3 in the process, and then access the site.

[0087] Furthermore, before the computing device can unmap the page, it also needs to interact with the device-side equipment. Specifically, the computing device first sends a demapping request to the heterogeneous accelerator card (corresponding to...). Figure 4Step S4 in the process. Then, the heterogeneous accelerator card updates the data in its cached address translation results (corresponding to...). Figure 4 In step S5), and after the update, requests involving access to the target data are centrally drained (corresponding to...). Figure 4 (Step S6 in the process). Then, upon confirming the relevant memory access request, the heterogeneous accelerator card sends a demapping response (corresponding to...) to the computing device. Figure 4 Step S7 in the process. Finally, the computing device can perform the demapping operation (corresponding to...). Figure 4 Step S8 in the process.

[0088] The aforementioned cumbersome interaction process makes the memory access process overly redundant and complex. Multiple interactions not only consume valuable I / O bandwidth between the heterogeneous accelerator card and the computing device, increasing communication overhead, but also reduce the performance of the heterogeneous accelerator card and the computing device.

[0089] In view of this, this application proposes a memory access method that uses a mapping state, a pre-demapping state, and a demapping state to characterize the mapping state between the target data that the heterogeneous accelerator card needs to access and the physical memory of the computing device. For a period of time before the computing device performs demapping, the state information corresponding to the target data is in the pre-demapping state, so that the heterogeneous accelerator card can know in time that the computing device is about to perform a demapping operation, and intervene in advance if necessary (if access is still required, the computing device is not allowed to demapping).

[0090] This not only avoids page faults, but also simplifies the memory access process and reduces communication overhead by eliminating the need for additional interaction between the heterogeneous accelerator card and the computing device before operations such as demapping. This results in better performance of the heterogeneous accelerator card and improved memory access efficiency.

[0091] like Figure 5 As shown, Figure 5 This application provides an embodiment of an interaction diagram illustrating the access of a heterogeneous accelerator card to the memory of a computing device. Figure 5 In this system, the computing device (host side) and the heterogeneous accelerator card (device side) each maintain a mapping relationship set (which records address information and corresponding status information). The synchronization process between the two mapping relationship sets can be initiated by one or more of the computing device and the heterogeneous accelerator card.

[0092] After receiving a memory access request, the heterogeneous accelerator card determines the mapping status between the target data corresponding to the request and the physical memory of the computing device through its maintained mapping relationship set, and then processes the memory access request accordingly (such as accessing the target data in physical memory) based on the mapping status.

[0093] In some feasible embodiments, this method is applied to a heterogeneous accelerator card, which is inserted into a computing device including memory and running virtual instances. A pass-through channel is established between the heterogeneous accelerator card and the virtual instances. The method specifically includes: receiving a memory access request sent by the virtual instance through the pass-through channel; responding to the memory access request by obtaining target status information corresponding to the target address; wherein the memory access request carries the target address and is used to request access to the target storage space in the memory corresponding to the target address; if the target status information indicates a pre-demapping state, sending a first instruction to the computing device; the pre-demapping state indicates that the computing device is about to de-map the established target mapping relationship between the target address and the target storage space; the first instruction is used to indicate that the computing device is not allowed to de-map the target; and accessing the target storage space through the target address. Since the pre-demapping state is a transitional state between the mapping state and the demapping state, if the heterogeneous accelerator card finds that the target status information corresponding to the target address is in a pre-demapping state before accessing memory, it can avoid page faults by sending a first instruction to indicate that the computing device is not allowed to de-map the target, thereby improving system reliability. Furthermore, heterogeneous accelerator cards eliminate the need for additional interaction with computing devices, simplifying memory access, reducing memory access overhead, and improving memory access efficiency.

[0094] Secondly, the system architecture of the embodiments of this application will be described by way of example.

[0095] Figure 6 This is a schematic diagram of the architecture of a cloud computing system provided in an embodiment of this application. Figure 6 As shown, the cloud computing system 100 includes a computing server cluster 110, a storage server cluster 120, a management server cluster 130, a network device cluster 140, and a user terminal 150. The computing server cluster 110, storage server cluster 120, and management server cluster 130 communicate with the user terminal 150 through the network device cluster 140.

[0096] Computing server cluster 110 includes one or more computing servers ( Figure 6 The diagram shows two computing servers, namely computing server 111 and computing server 112 (but is not limited to two computing servers). As computing resources within the cloud computing system 100, such as servers or desktop computers, computing servers can be used to deploy virtual instances (such as virtual machines) and allocate corresponding computing resources based on virtualization technology and user needs. At the hardware level, computing servers are equipped with processors and memory; the computing functions of the computing server are implemented by the processor running programs in memory.

[0097] Storage server cluster 120 includes one or more storage servers ( Figure 6The diagram shows two storage servers, namely storage server 121 and storage server 122, but is not limited to two storage servers. The storage servers, as storage resources in the cloud computing system 100, such as servers, desktop computers, or controllers and hard disk enclosures of storage arrays, are used to provide logical disk storage and integrated backup services for virtual instances within the cloud computing system 100.

[0098] Management server cluster 130 includes one or more management servers ( Figure 6 The diagram shows two management servers, namely management server 131 and management server 132, but is not limited to two management servers. The management servers are used to manage all computing services, shared storage, and network of the entire cloud computing system 100, and at the same time provide users or administrators with an application program interface (API) for managing the entire node.

[0099] Network device cluster 140 includes one or more switches and routers, such as Figure 6 As shown, in this embodiment, the network device cluster 140 includes a router 141, a switch 142, a switch 143, a switch 144, and a switch 145. The user terminal 150 is connected to the router 141 via the Internet. The router 141 is then connected to the switch 143 via the switch 142. The switch 143 is connected to each computing server in the computing server cluster 110. The switch 144 is connected to each computing server in the computing server cluster 110 and each storage server in the storage server cluster 120. The switch 145 is connected to each computing server in the computing server cluster 110, each storage server in the storage server cluster 120, and each management server in the management server cluster 130.

[0100] Optionally, the number and type of switches included in the network device cluster 140 can be adjusted according to the needs of the cloud computing system 100. Switches 142, 143, 144 and 145 can be switches with different functions, and this application does not impose any restrictions on this.

[0101] User terminal 150 includes one or more user terminals ( Figure 6 The diagram shows two user terminals, namely user terminal 151 and user terminal 152, but is not limited to two user terminals. The user terminals contain the interfaces and applications required for accessing the cloud computing system 100.

[0102] It is worth noting that, Figure 6 This is merely an illustration and should not be construed as limiting the scope of this application. The cloud computing system 100 may also include other devices. Figure 6 It is not shown in the middle.

[0103] The computing servers, storage servers, etc., in the cloud computing system 100 provided in this application can be nodes in a cloud platform. For example... Figure 6 Based on the equipment of the cloud computing system 100 shown, the cloud computing system 100 implements the functions of nodes such as computing nodes and storage nodes based on Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS), and provides services (such as computing services, storage services, and network services) to user terminals 150 through these nodes. These nodes can be service nodes (such as computing nodes and storage nodes) in the cloud obtained by virtualizing the resources (such as computing resources and storage resources) of the cloud computing system 100.

[0104] The memory access method provided in this application embodiment can be applied to... Figure 7 The heterogeneous accelerator card shown can be inserted into... Figure 6 Any one of the servers (computing devices) can be used to perform some of the computing tasks that would otherwise be performed by the server CPU (such as network communication), thereby improving overall performance.

[0105] Figure 7 This is a schematic diagram of a heterogeneous accelerator card provided in an embodiment of this application. Figure 7 The heterogeneous accelerator card 3000 shown includes at least: a memory 3010, a processor 3020, and a bus 3030.

[0106] The processor 3020 is at least capable of obtaining target status information corresponding to the target address in response to a memory access request, and performing appropriate processing based on the target status information (such as sending a first instruction to the computing device when the target status information indicates a pre-demapping state) to avoid page faults. The memory 3010 is at least capable of storing the logic code corresponding to the memory access method provided in the embodiments of this application.

[0107] Optionally, the heterogeneous accelerator card 3000 can be a DPU (including but not limited to SmartNIC), NPU, GPU, ASIC, FPGA, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Correspondingly, the processor 3020 is the hardware computing unit therein.

[0108] Optionally, the memory 3010 may include random access memory (RAM), read-only memory (ROM), etc., wherein the memory 3010 may run the necessary operating system in its RAM, as well as modules such as an acquisition module and an allocation module for executing the memory access method provided in this application.

[0109] Optionally, the bus 3030 can be a PCIe bus, etc., and this application does not limit the type of bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 7 The bus 3030 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 3030 may include a path for transmitting information between the various components of the heterogeneous accelerator card 3000 (e.g., memory 3010 and processor 3020).

[0110] It is worth noting that, Figure 7 The heterogeneous accelerator card shown can not only be inserted into Figure 6 The server in the cloud computing system 100 can also be placed on any server in other systems or application scenarios and execute the memory access method provided in the embodiments of this application. This application does not limit this.

[0111] The system architecture and application scenarios described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0112] For ease of understanding, the memory access method provided in the embodiments of this application is described below with reference to the accompanying drawings. This memory access method is applicable to... Figure 7 The heterogeneous accelerator card shown is inserted into a computing device, which includes memory and runs virtual instances. A pass-through channel is established between the heterogeneous accelerator card and the virtual instances. In this embodiment, the virtual instances can be virtual machines, containers, container groups (pods), bare metal servers, etc., utilizing the server's processor and memory resources through computing virtualization technology.

[0113] See details Figure 8 , Figure 8 An interactive flowchart of a memory access method provided in this application embodiment specifically includes the following steps:

[0114] First, before the heterogeneous accelerator card accesses memory, the following steps may typically be included:

[0115] S110, Initialize the mapping relationship set for the computing device.

[0116] In this step, the computing device can allocate a corresponding amount of memory according to the usage requirements of the virtual instance, establish a mapping relationship between the memory address space of the computing device and the memory address space of the virtual instance, and record the aforementioned mapping relationship by initializing the mapping relationship set (which can be a page table or a subset of page tables).

[0117] See Table 1 below, which is a possible example of a set of mapping relations:

[0118] Table 1

[0119] First field Second field ··· physical address

[0120] Table 1 records the physical addresses of the memory allocated to virtual instances in physical memory, as well as the correspondence between multiple fields. These fields can be used to represent access attributes, read and write permissions, and status information, etc. The address in the address space of the virtual instance corresponding to each physical address can be used as the index value of each entry in the table to index the corresponding physical address.

[0121] In this process, after the computing device has just created the mapping set, it can indicate that the physical address in this mapping set is invalid by setting the values ​​of one or more fields. Each field can occupy one or more bits. For example, the first field (occupying at least one bit) can be set to 0, and the second field (occupying at least one bit) can also be set to 0.

[0122] After the mapping relationship is established for the computing devices, the corresponding values ​​in the mapping relationship set can be modified to indicate that the corresponding physical address can be accessed normally. For example, the first field can be set to 1, while the second field remains at 0.

[0123] After initialization, the computing device can also send the mapping relationship set obtained after initialization (which can be referred to as the host-side mapping relationship set or the second mapping relationship set in this embodiment) to the target heterogeneous acceleration card. Alternatively, the computing device can also store the mapping relationship set in a set location (from which the target heterogeneous acceleration card can read data). The aforementioned target heterogeneous acceleration card refers to a heterogeneous acceleration card that is inserted into the computing device and has a direct communication channel established with the aforementioned virtual instance (or, a heterogeneous acceleration card that the virtual instance can use through hardware pass-through).

[0124] See Table 2 below, which is a possible example of the initialized mapping set:

[0125] Table 2

[0126] First field Second field ··· physical address 1 0 Address 1

[0127] Table 2 shows only one row of data as an example. Address 1 in Table 2 could be any one of the initialized mapping relationships. The first and second fields can each contain at least one bit. The mapping state, pre-demapping state, and demapping state are represented by the permutations and combinations of two bits (a total of 4 possibilities). The specific correspondence can be set according to the actual situation.

[0128] Alternatively, see Table 3 below, which provides another possible example of the initialized mapping set:

[0129] Table 3

[0130] First field ··· physical address 100 Address 2

[0131] Table 3 shows only one row of data as an example. Address 2 in Table 3 could be any address in the initialized mapping set. The first field, by occupying multiple bits of data, can have a wider range of values. Furthermore, different values ​​of the first field can represent the mapped state, pre-demapped state, and demapped state (or, it can be understood as representing not only three discrete states but also a continuous state evolution process). The corresponding state information is determined by comparing it with the set value range; the specific correspondence can be set according to the actual situation.

[0132] Furthermore, given the initialized mapping set as shown in Table 3, the value of the first field can continuously decrease over time (e.g., decrease by 10 per second) and increase when the heterogeneous accelerator card accesses address 2 (e.g., increase by 1 with each access). Thus, over a period of time, the first field can also be understood as indicating the popularity of the data stored at address 2.

[0133] S120, heterogeneous accelerator card obtains mapping relationship set.

[0134] In this step, the heterogeneous acceleration card may respond to receiving the host-side mapping relationship set sent by the computing device, or periodically detect whether data exists at a set location to obtain the latest host-side mapping relationship set and store it in its own storage space to obtain the device-side mapping relationship set (first mapping relationship set), thereby completing step S120.

[0135] Based on steps S110-S120, the computing device and the heterogeneous accelerator card each begin to maintain a mapping relationship set. The heterogeneous accelerator card can directly access the memory of the computing device through the mapping relationship set (without the CPU of the computing device participating), while the computing device can monitor the status of each address according to the mapping relationship set to complete resource scheduling more accurately.

[0136] S130, the synchronized modified mapping set.

[0137] In this step, both the computing device and the heterogeneous accelerator card can modify their respective maintained mapping relationship sets according to their actual status and synchronize them to the other side in a timely manner.

[0138] Specifically, see Figure 9 , Figure 9 This application provides a schematic diagram comparing the state information before and after the improvement. Unlike traditional methods, where a computing device or heterogeneous accelerator card can only change the state information from the mapped state (target data can be accessed normally) to the demapped state (target data cannot be accessed normally) when modifying the state information of a certain address in the mapping relationship set, these are two completely opposite states. Figure 9 (Left side)

[0139] In this embodiment of the application, in addition to the mapping state and the demapping state, at least a third state is added: the pre-demapping state. Figure 9 (On the right), this state is between the mapped state and the demapped state. When the state information corresponding to the address is in the pre-demapped state, the heterogeneous accelerator card can still access the corresponding storage space in memory normally, but further processing measures are required, such as... Figure 9 The trigger command is sent.

[0140] In this embodiment of the application, the mapping state can instruct the computing device to maintain the established mapping relationship between the corresponding address and the storage space in memory; the pre-demapping state can instruct the computing device to release the established target mapping relationship between the address and the storage space in memory; and the demapping state can instruct the computing device to release the mapping relationship.

[0141] And from Figure 9 As can be seen intuitively, the improved mapping state does not jump directly to the demapping state, but transitions through the pre-demapping state (the transition time is not limited), which allows heterogeneous accelerator cards and computing devices to know the more detailed state of the target data in advance, thus avoiding page fault problems.

[0142] Modifying the state information corresponding to each address in the mapping set can be done to improve resource utilization. For example, a computing device may only have 24GB of memory, but through memory super-division and other technologies, it can run three virtual instances, each requiring 12GB of memory (meaning a total of 36GB of memory). Therefore, the computing device needs to dynamically schedule the memory usage of each virtual instance.

[0143] For example, when a certain allocated storage space in memory is not frequently accessed (access frequency is below a threshold), the computing device first modifies the state information corresponding to the address of this storage space in the mapping relationship set (specifically, it modifies it to the pre-demapping state) and synchronizes it with the heterogeneous accelerator card. If it does not receive an instruction from the heterogeneous accelerator card that it does not allow the mapping relationship to be de-mapped, it de-maps the relationship (the source data can be swapped to the disk) and establishes a new mapping relationship, thereby realizing the reallocation of memory.

[0144] Specifically, a field in the mapping relationship set can be used to record the frequency of accessing the corresponding storage space in memory through the address in the mapping relationship set (as shown in the first field of Table 3 above). When the frequency meets the target range, the status information indicates the pre-demapping state; when the frequency is higher than the target range, the status information indicates the mapping state; and when the frequency is lower than the target range, the status information indicates the demapping state. Here, "higher than the target range" specifically means higher than the maximum value within the target range, and "lower than the target range" specifically means lower than the minimum value within the target range.

[0145] For example, a computing device determines a target range of 80-100 times / second by counting the access frequency of all allocated storage space in its memory. The computing device detects that the access frequency of a storage space has decreased from 110 times / second to 90 times / second in a recent period.

[0146] Then, the computing device modifies the state information of the storage space (corresponding to address 3) in the host-side mapping relationship set, specifically from the mapped state to the pre-demapped state, and sends a synchronization command to the heterogeneous accelerator card, instructing the heterogeneous accelerator card to synchronize the state indicated by the target state information corresponding to address 3 to the pre-demapped state.

[0147] Then, in response to receiving the synchronization command sent by the computing device, the heterogeneous accelerator card synchronizes the state indicated by the state information of address 3 in the device-side mapping relationship set to the pre-demapping state.

[0148] It is worth noting that the modification and synchronization process of state information involved in step S130 can also be repeated and executed in parallel in subsequent steps. Steps S110-S170 shown here are only one possible example.

[0149] The S140 heterogeneous accelerator card receives memory access requests sent by virtual instances through a pass-through channel.

[0150] The memory access request carries a target address, used to request access to the target storage space in memory corresponding to that target address. For example, the target address carried in the memory access request can be an input / output virtual address (IOVA). Furthermore, if the heterogeneous accelerator card stores a device-side mapping set, the IOVA can be used as an index value to determine the target status information corresponding to the device-side mapping set.

[0151] It's worth noting that the pass-through between the virtual instance and the heterogeneous accelerator card bypasses the hypervisor's monitoring (while also achieving better performance). This means the hypervisor cannot detect the specific communication behavior on the pass-through, and therefore cannot know that the virtual instance sent a memory access request to the heterogeneous accelerator card. In this situation, the hypervisor struggles to effectively act as a communication bridge between the virtual instance, the host machine hosting the virtual instance, and the heterogeneous accelerator card.

[0152] S150, in response to a memory access request, the heterogeneous accelerator card obtains the target status information corresponding to the target address.

[0153] Specifically, after receiving a memory access request sent by a virtual instance, the heterogeneous accelerator card can at least obtain the target status information corresponding to the target address carried in the memory access request by querying the device-side mapping relationship set (the first mapping relationship set).

[0154] For example, in response to a memory access request, the heterogeneous accelerator card queries the device-side mapping relationship set (the specific storage format can be referred to Table 1 or Table 2 above) to obtain the target status information indicating the target address (address 3) as the pre-demapping state.

[0155] S160, when the target state information indicates a pre-demapping state, the heterogeneous accelerator card sends a first instruction to the computing device.

[0156] The pre-demapping state, as mentioned above, indicates that the computing device is about to de-map the target address and the target storage space. However, the heterogeneous accelerator card still has memory access requests that need to access the data. Therefore, it sends a first instruction to the computing device, instructing it not to de-map the target.

[0157] Specifically, the first instruction sent by the heterogeneous accelerator card may be an instruction to prevent the computing device from releasing the target mapping relationship within a set time. The set time may be determined based on the average time taken for the heterogeneous accelerator card to complete a memory access request, and this application does not impose any restrictions on it.

[0158] In this way, the heterogeneous accelerator card extends the target mapping relationship through the first instruction, thereby enabling it to complete the memory access requests sent by the virtual instance normally and avoiding page faults.

[0159] S170, in response to the first instruction, the computing device does not immediately remove the target mapping relationship.

[0160] After receiving the first instruction sent by the heterogeneous accelerator card, the computing device will not release the target mapping relationship within a set time, ensuring that the heterogeneous accelerator card can access the target data, complete the memory access request of the virtual instance, and meet the needs of related business operations.

[0161] S180, heterogeneous accelerator cards access target storage space through target addresses.

[0162] It is worth noting that step S170 is not necessarily executed only after steps S150-S160. In some feasible embodiments, the target storage space can be accessed first after receiving the memory access request, and then the first instruction can be sent. This application does not limit this.

[0163] Heterogeneous accelerator cards access target storage space through target addresses, where access can include both read and write access, and this application does not impose any restrictions on this.

[0164] By using steps S110-S180 above, the mapping status between the target data to be accessed by the heterogeneous accelerator card and the physical memory of the computing device is characterized by two or more types of status information, thus avoiding page faults. Combined with the normal status information synchronization process, no additional interaction is required before the heterogeneous accelerator card accesses memory or before the computing device de-mappings the relationship, simplifying the memory access process and reducing memory access overhead.

[0165] To lower the deployment threshold, the following example uses only one bit for each of the first field (valid bit) and the second field (eviction bit). That is, at least four different permutations and combinations can be generated using at least two single-bit flags, enabling the convenient implementation of the memory access method provided in this application's embodiments.

[0166] See details Figure 10 , Figure 10 A schematic diagram illustrating a change in status information provided in an embodiment of this application:

[0167] Typically, when a computing device has just created a set of memory allocation mappings (which may be page tables), taking one of the addresses involved as an example, its corresponding status information is usually: valid bit: 0 and eviction bit: 0, or it can be abbreviated as: (0, 0), indicating a demapping state, or an invalid state.

[0168] Subsequently, after the computing device completes the mapping, the status information corresponding to the address is changed from (0,0) to (1,0), indicating a mapped state, at which point the heterogeneous accelerator card can access it normally.

[0169] Then, as time goes on (a time span can be set), the frequency of access to each storage space may differ. At this time, the computing device can maintain the status information (1, 0) for access frequencies that are normal (above the set frequency range); while modifying the status information (1, 1) for access frequencies that are low (within the set frequency range) from (0, 0) to (1, 1), indicating a pre-demapping state. That is to say, the computing device may demapping this part of the mapping relationship in the future.

[0170] If the heterogeneous accelerator card needs to access the target data corresponding to the status information (1,1), it will trigger an instruction to instruct the computing device not to remove the target mapping relationship corresponding to the target data, thus prolonging the maintenance time of the target mapping relationship.

[0171] Again, as time progresses (a different or the same time span can be set), the computing device, based on the latest access frequency of each storage space, changes the status information of storage spaces with low access frequency (below the set frequency range) from (1, 1) to (0, 1), indicating a demapping state. This means the computing device has removed this mapping relationship and does not allow heterogeneous accelerator cards to directly access it. To further reduce the possibility of page faults, the computing device synchronizes with the heterogeneous accelerator cards before removing this mapping relationship, ensuring that the heterogeneous accelerator cards will not directly access this data, and then removes the mapping relationship.

[0172] It is worth noting that, in the aforementioned process, the modification of the status information can be initiated by the computing device and synchronized to the heterogeneous accelerator card, or it can be initiated by the heterogeneous accelerator card and then synchronized to the computing device.

[0173] The foregoing mainly describes the solutions of the embodiments of this application from the perspective of methods and module interactions. It is understood that, in order to achieve the above functions, the memory access device includes at least one of the hardware structures and software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software-driven hardware manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0174] This application embodiment can divide the memory access device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. The unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0175] For example, Figure 11 This is a schematic diagram of a memory access device provided in an embodiment of this application. The memory access device 800 is applied in a heterogeneous accelerator card, or the memory access device 800 can be a heterogeneous accelerator card itself. The aforementioned heterogeneous accelerator card is inserted into a computing device, which includes memory and runs virtual instances. A direct communication channel is established between the memory access device 800 and the virtual instances. The device 800 includes:

[0176] The receiving module 810 is used to receive memory access requests sent by virtual instances through a pass-through channel;

[0177] The acquisition module 820 is used to obtain target status information corresponding to the target address in response to a memory access request; wherein, the memory access request carries the target address and is used to request access to the target storage space in the memory corresponding to the target address;

[0178] The sending module 830 is used to send a first instruction to the computing device when the target status information indicates a pre-demapping state; the pre-demapping state indicates that the computing device is about to de-map the established target mapping relationship between the target address and the target storage space; the first instruction is used to indicate that the computing device is not allowed to de-map the target relationship.

[0179] Access module 840 is used to access the target storage space via the target address.

[0180] In one possible implementation, the acquisition module 820 is further configured to, in response to a memory access request, query a first mapping set to obtain target status information corresponding to the target address; wherein the first mapping set is stored in a heterogeneous accelerator card, and the first mapping set includes at least one address and its corresponding status information, and the at least one address includes the physical address of the target storage space in memory.

[0181] In one possible implementation, in the first mapping relationship set, state information is carried through at least one field, indicating a mapping state, a demapping state, or a pre-demapping state; wherein, the mapping state indicates that the computing device maintains an established mapping relationship between at least one address and a storage space in memory; the demapping state indicates that the computing device has demapped the relationship.

[0182] In one possible implementation, the status information is the frequency of accessing the corresponding storage space in memory through the address in the first mapping relationship set; when the frequency meets the target range, the status information indicates the pre-demapping state; when the frequency is higher than the target range, the status information indicates the mapping state; when the frequency is lower than the target range, the status information indicates the demapping state.

[0183] In one possible implementation, the memory access device 800 further includes a modification module, which is configured to modify target status information if, after accessing the target storage space via the target address, the number of times the target storage space is accessed via the target address within a first set time period is less than a first threshold; wherein the modified target status information indicates a demapping state; the demapping state indicates that the computing device has released the target mapping relationship; or, if, within a second set time period, the number of times the target storage space is accessed via the target address is greater than a second threshold, the target status information is modified; wherein the modified target status information indicates a mapping state; the mapping state indicates that the computing device maintains the target mapping relationship.

[0184] In one possible implementation, the memory access device 800 further includes a first synchronization module, which is used to synchronize the modified target state information to the computing device after modifying the target state information.

[0185] In one possible implementation, the memory access device 800 further includes a second synchronization module, which is used to receive a synchronization instruction sent by the computing device before obtaining the target status information corresponding to the target address in response to the memory access request. The synchronization instruction is used to instruct the state indicated by the target status information corresponding to the target address to be synchronized to the pre-demapping state.

[0186] For example, combining Figure 8 The acquisition module 820 can be used to perform actions such as... Figure 8 As shown in S120, the transmitting module 830 can be used to perform, for example... Figure 8 As shown in S150, the access module 840 can be used to perform, for example... Figure 8 S170 is shown.

[0187] For example, Figure 12 This is a schematic diagram of another memory access device provided in an embodiment of this application. The memory access device 900 is applied in a computing device, or the memory access device 900 can be a computing device, the aforementioned computing device including memory, and a heterogeneous accelerator card inserted in the computing device, with a virtual instance running on the computing device, and a pass-through channel established between the virtual instance and the heterogeneous accelerator card; the memory access device 900 includes:

[0188] The receiving module 910 is used to receive a first instruction sent by the heterogeneous accelerator card. The first instruction is used to indicate that the target mapping relationship between the target address and the target status information is not allowed to be removed. The target address indicates a pre-demapping state. The pre-demapping state indicates that the computing device is about to remove the established target mapping relationship between the target address and the target storage space.

[0189] The release module 920 is used to temporarily not release the target mapping relationship in response to the first instruction.

[0190] In one possible implementation, the computing device stores a second mapping set, in which state information is carried by at least one field, indicating a mapping state, a demapping state, or a pre-demapping state; wherein, the mapping state indicates that the computing device maintains an established mapping relationship between at least one address and a storage space in memory; the demapping state indicates that the computing device has demapped the relationship.

[0191] In one possible implementation, the status information is the frequency of the heterogeneous accelerator card accessing the corresponding storage space in memory; when the frequency meets the target range, the status information indicates the pre-demapping state; when the frequency is higher than the target range, the status information indicates the mapping state; when the frequency is lower than the target range, the status information indicates the demapping state.

[0192] In one possible implementation, the memory access device 900 further includes a modification module, which is used to count the frequency of heterogeneous accelerator cards accessing the target storage space; if the frequency is lower than a first threshold, modify the target status information; wherein the modified target status information indicates a demapping state; the demapping state indicates that the computing device has released the target mapping relationship; or, if the frequency is higher than a second threshold, modify the target status information; wherein the modified target status information indicates a mapping state; the mapping state indicates that the computing device maintains the target mapping relationship.

[0193] In one possible implementation, the memory access device 900 further includes a second synchronization module, which is used to synchronize the modified target state information to the heterogeneous accelerator card after updating the target state information according to the frequency.

[0194] In one possible implementation, the memory access device 900 further includes a second modification module, which is used to receive a synchronization instruction sent by the heterogeneous acceleration card; the synchronization instruction is used to instruct the state indicated by the target state information corresponding to the target address to be synchronized to the pre-demapping state; and to synchronize the state indicated by the target state information to the pre-demapping state.

[0195] As a feasible example, the memory access device 800 or 900 provided in this application is implemented through a software module. For example, the software module can be provided to users through a cloud service subscription model, and users can choose different subscription levels according to their needs. Alternatively, the software module can also provide enterprise-level customized services with professional domain customization, interface personalization and extended functions according to the needs of users or enterprises.

[0196] Furthermore, the memory access device 800 or 900 provided in this application can also be provided to users as a value-added service, and this application does not limit this. When the memory access device 800 or 900 is implemented through a software module, the memory access device 800 or 900 can also be embedded into memory access management software or cloud service resource allocation system.

[0197] This application also provides a computing device 100. For example... Figure 13 As shown, the computing device 100 includes a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate with each other via the bus 102. The computing device 100 may be a server or a terminal device. It should be understood that the embodiments of this application do not limit the number of processors and memories in the computing device 100.

[0198] Bus 102 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 13 The bus 102 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 102 may include a path for transmitting information between various components of the computing device 100 (e.g., memory 106, processor 104, communication interface 108).

[0199] The processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0200] Memory 106 may include volatile memory, such as random access memory (RAM). Processor 104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0201] The memory 106 stores executable program code, and the processor 104 executes the executable program code to implement the functions of the aforementioned receiving module and releasing module, thereby realizing the memory access method. That is, the memory 106 stores instructions for executing the memory access method.

[0202] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0203] like Figure 14 As shown, the computing device cluster includes at least one computing device 100. The memory 106 of one or more computing devices 100 in the computing device cluster may store the same instructions for executing memory access methods.

[0204] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing memory access methods. In other words, a combination of one or more computing devices 100 can jointly execute instructions for executing memory access methods.

[0205] The memory 106 in different computing devices 100 within the computing device cluster can store different instructions, each used to execute a portion of the functions of the memory access device. That is, the instructions stored in the memory 106 of different computing devices 100 can implement the functions of one or more modules, such as the receive module and the release module.

[0206] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 15 One possible implementation is shown. For example... Figure 15As shown, two computing devices 100A and 100B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this possible implementation, the memory 106 in computing device 100A stores instructions for performing the functions of the receiving module. Simultaneously, the memory 106 in computing device 100B stores instructions for performing the functions of the releasing module.

[0207] It should be understood that Figure 15 The functions of the computing device 100A shown can also be performed by multiple computing devices 100. Similarly, the functions of the computing device 100B can also be performed by multiple computing devices 100.

[0208] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform a memory access method.

[0209] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform a memory access method, or instruct the computing device to perform a memory access method.

[0210] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A memory access method, characterized in that, The application is to a heterogeneous accelerator card, which is inserted into a computing device, the computing device including memory, and a virtual instance running on the computing device, and a pass-through channel is established between the heterogeneous accelerator card and the virtual instance; The method includes: Receive memory access requests sent by the virtual instance through the direct channel; In response to the memory access request, target status information corresponding to the target address is obtained; wherein, the memory access request carries the target address and is used to request access to the target storage space in the memory corresponding to the target address; When the target status information indicates a pre-demapping state, a first instruction is sent to the computing device; the pre-demapping state indicates that the computing device is about to terminate the established target mapping relationship between the target address and the target storage space; the first instruction is used to indicate that the computing device is not allowed to terminate the target mapping relationship. Access the target storage space through the target address.

2. The method according to claim 1, characterized in that, The step of obtaining the target status information corresponding to the target address in response to the memory access request includes: In response to the memory access request, a first mapping set is queried to obtain the target status information corresponding to the target address; wherein, the first mapping set is stored in the heterogeneous acceleration card, and the first mapping set includes at least one address and its corresponding status information, the at least one address including the physical address of the target storage space in the memory.

3. The method according to claim 2, characterized in that, In the first mapping relationship set, state information is carried through at least one field, and the state information indicates the mapping state, demapping state, or pre-demapping state. The mapping state indicates that the computing device maintains the established mapping relationship between the at least one address and the storage space in the memory; the demapping state indicates that the computing device has released the mapping relationship.

4. The method according to claim 3, characterized in that, The status information is the frequency of accessing the corresponding storage space in the memory through the address in the first mapping relationship set; When the frequency meets the target range, the status information indicates the pre-demapping state; When the frequency is higher than the target range, the status information indicates a mapping state; When the frequency is lower than the target range, the status information indicates a demapping state.

5. The method according to any one of claims 1-4, characterized in that, After accessing the target storage space via the target address, the method further includes: If the number of times the target storage space is accessed through the target address within a first predetermined time period is less than a first threshold, the target status information is modified; wherein, the modified target status information indicates a demapping state; the demapping state indicates that the computing device has released the target mapping relationship; or... If the number of times the target storage space is accessed through the target address within a second set time period exceeds a second threshold, the target status information is modified; wherein, the modified target status information indicates a mapping state; the mapping state indicates that the computing device maintains the target mapping relationship.

6. The method according to claim 5, characterized in that, After modifying the target state information, the method further includes: The modified target status information is synchronized to the computing device.

7. The method according to any one of claims 1-6, characterized in that, Before obtaining the target status information corresponding to the target address in response to the memory access request, the method further includes: The system receives a synchronization instruction sent by the computing device, the synchronization instruction being used to instruct the target state information corresponding to the target address to be synchronized to the pre-demapping state; The state indicated by the target state information is synchronized to the pre-demapped state.

8. A memory access method, characterized in that, The method is applied to a computing device, which includes memory and interpolates a heterogeneous accelerator card on the computing device. A virtual instance runs on the computing device, and a pass-through channel is established between the heterogeneous accelerator card and the virtual instance. The method includes: The device receives a first instruction sent by the heterogeneous acceleration card. The first instruction is used to indicate that the target mapping relationship between the target address and the target status information is not allowed to be removed. The target address indicates a pre-demapping state. The pre-demapping state indicates that the computing device is about to remove the established target mapping relationship between the target address and the target storage space. In response to the first instruction, the target mapping relationship is not terminated for the time being.

9. The method according to claim 8, characterized in that, The computing device stores a second mapping relationship set, in which state information is carried by at least one field, the state information indicating the mapping state, the demapping state, or the pre-demapping state; The mapping state indicates that the computing device maintains the established mapping relationship between the at least one address and the storage space in the memory; the demapping state indicates that the computing device has released the mapping relationship.

10. The method according to claim 9, characterized in that, The status information is the frequency with which the heterogeneous accelerator card accesses the corresponding storage space in the memory; When the frequency meets the target range, the status information indicates the pre-demapping state; When the frequency is higher than the target range, the status information indicates a mapping state; When the frequency is lower than the target range, the status information indicates a demapping state.

11. The method according to any one of claims 8-10, characterized in that, The method further includes: Statistically count the frequency of the heterogeneous accelerator card accessing the target storage space; If the frequency is lower than a first threshold, the target state information is modified; wherein the modified target state information indicates a demapping state; the demapping state indicates that the computing device has removed the target mapping relationship; or... If the frequency is higher than a second threshold, the target state information is modified; wherein the modified target state information indicates a mapping state; the mapping state indicates that the computing device maintains the target mapping relationship.

12. The method according to claim 11, characterized in that, After updating the target state information according to the frequency, the method further includes: The modified target status information is synchronized to the heterogeneous acceleration card.

13. The method according to any one of claims 8-12, characterized in that, The method further includes: Receive a synchronization command sent by the heterogeneous acceleration card; the synchronization command is used to instruct the target state information corresponding to the target address to be synchronized to the pre-demapping state; The state indicated by the target state information is synchronized to the pre-demapped state.

14. A memory access device, characterized in that, The device is applied to a heterogeneous accelerator card, which is inserted into a computing device. The computing device includes memory and runs a virtual instance. A pass-through channel is established between the heterogeneous accelerator card and the virtual instance. The device includes: A receiving module is configured to receive memory access requests sent by the virtual instance through the direct channel; The acquisition module is used to acquire target status information corresponding to the target address in response to the memory access request; wherein the memory access request carries the target address and is used to request access to the target storage space in the memory corresponding to the target address; The sending module is configured to send a first instruction to the computing device when the target status information indicates a pre-demapping state; the pre-demapping state indicates that the computing device is about to terminate the established target mapping relationship between the target address and the target storage space; the first instruction is used to indicate that the computing device is not allowed to terminate the target mapping relationship. An access module is used to access the target storage space via the target address.

15. The apparatus according to claim 14, characterized in that, The acquisition module is also used for, In response to the memory access request, a first mapping set is queried to obtain the target status information corresponding to the target address; wherein, the first mapping set is stored in the heterogeneous acceleration card, and the first mapping set includes at least one address and its corresponding status information, the at least one address including the physical address of the target storage space in the memory.

16. The apparatus according to claim 15, characterized in that, In the first mapping relationship set, state information is carried through at least one field, and the state information indicates the mapping state, demapping state, or pre-demapping state. The mapping state indicates that the computing device maintains the established mapping relationship between the at least one address and the storage space in the memory; the demapping state indicates that the computing device has released the mapping relationship.

17. The apparatus according to claim 16, characterized in that, The status information is the frequency of accessing the corresponding storage space in the memory through the address in the first mapping relationship set; When the frequency meets the target range, the status information indicates the pre-demapping state; When the frequency is higher than the target range, the status information indicates a mapping state; When the frequency is lower than the target range, the status information indicates a demapping state.

18. The apparatus according to any one of claims 14-17, characterized in that, The device further includes a modification module, the modification module being used to, After accessing the target storage space via the target address, if the number of times the target storage space is accessed via the target address within a first set time period is less than a first threshold, the target status information is modified; wherein, the modified target status information indicates a demapping state; the demapping state indicates that the computing device has released the target mapping relationship; or... If the number of times the target storage space is accessed through the target address within a second set time period exceeds a second threshold, the target status information is modified; wherein, the modified target status information indicates a mapping state; the mapping state indicates that the computing device maintains the target mapping relationship.

19. The apparatus according to claim 18, characterized in that, The device further includes a first synchronization module, which is used to: After modifying the target state information, the modified target state information is synchronized to the computing device.

20. The apparatus according to any one of claims 16-19, characterized in that, The device further includes a second synchronization module, which is used for: Before obtaining the target status information corresponding to the target address in response to the memory access request, a synchronization instruction sent by the computing device is received. The synchronization instruction is used to instruct the state indicated by the target status information corresponding to the target address to be synchronized to the pre-demapping state. The state indicated by the target state information is synchronized to the pre-demapped state.

21. A memory access device, characterized in that, The device is applied to a computing device, which further includes memory, and a heterogeneous accelerator card is inserted into the computing device. A virtual instance runs on the computing device, and a pass-through channel is established between the heterogeneous accelerator card and the virtual instance. The device includes: The receiving module is used to receive a first instruction sent by the heterogeneous acceleration card. The first instruction is used to indicate that the target mapping relationship between the target address and the target status information is not allowed to be removed. The state indicated by the target address is a pre-demapping state. The pre-demapping state indicates that the computing device is about to remove the established target mapping relationship between the target address and the target storage space. The release module is used to temporarily not release the target mapping relationship in response to the first instruction.

22. The apparatus according to claim 21, characterized in that, The computing device stores a second mapping relationship set, in which state information is carried by at least one field, the state information indicating the mapping state, the demapping state, or the pre-demapping state; The mapping state indicates that the computing device maintains the established mapping relationship between the at least one address and the storage space in the memory; the demapping state indicates that the computing device has released the mapping relationship.

23. The apparatus according to claim 22, characterized in that, The status information is the frequency with which the heterogeneous accelerator card accesses the corresponding storage space in the memory; When the frequency meets the target range, the status information indicates the pre-demapping state; When the frequency is higher than the target range, the status information indicates a mapping state; When the frequency is lower than the target range, the status information indicates a demapping state.

24. The apparatus according to any one of claims 21-23, characterized in that, The device further includes a modification module, the modification module being used to, Statistically count the frequency of the heterogeneous accelerator card accessing the target storage space; If the frequency is lower than a first threshold, the target state information is modified; wherein the modified target state information indicates a demapping state; the demapping state indicates that the computing device has removed the target mapping relationship; or... If the frequency is higher than a second threshold, the target state information is modified; wherein the modified target state information indicates a mapping state; the mapping state indicates that the computing device maintains the target mapping relationship.

25. The apparatus according to claim 24, characterized in that, The device further includes a second synchronization module, which is used for: After updating the target status information according to the frequency, the modified target status information is synchronized to the heterogeneous acceleration card.

26. The apparatus according to any one of claims 21-25, characterized in that, The device further includes a second modification module, the second modification module being used for, Receive a synchronization command sent by the heterogeneous acceleration card; the synchronization command is used to instruct the target state information corresponding to the target address to be synchronized to the pre-demapping state; The state indicated by the target state information is synchronized to the pre-demapped state.

27. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions; when the computer instructions are executed in the heterogeneous accelerator card, the heterogeneous accelerator card performs the memory access method according to any one of claims 1-7.

28. A computer program product, characterized in that, When the computer program product is run on a heterogeneous accelerator card, the heterogeneous accelerator card performs the memory access method according to any one of claims 1-7.

Images