A cross-host memory sharing method, system, device and medium

Abstract

The present application discloses a cross-host memory sharing method, system, device and medium. The memory sharing system includes a master node and several slave nodes. The memory sharing method is applied to the master node and relates to the field of memory sharing technology, including: receiving a memory application request from a target slave node; applying for a target memory in a target memory shared area according to the memory size requirement corresponding to the request, feeding back the memory information of the target memory to the target slave node so that the target slave node can determine the corresponding memory type according to the memory information. If the process corresponding to the target slave node executes a memory mapping method, the target memory can be mapped to the virtual address space according to the target type and the offset corresponding to the target memory so that the process can use the allocated target memory. In this way, it can be ensured that when one node writes to the memory, other nodes will not write to the memory synchronously, thereby ensuring the data consistency of multiple hosts accessing the shared memory.

Description

Technical Field

[0001] The present application relates to the field of memory sharing technology, and in particular to a cross-host memory sharing method, system, device and medium. Background Art

[0002] In artificial intelligence training, big data analysis, and cloud computing platforms, data interaction between multi-host clusters is becoming increasingly frequent. Although traditional network transmission solutions based on RDMA (Remote Direct Memory Access) reduce some latency by bypassing the operating system protocol stack, they are still limited by issues such as physical network topology, TCP (Transmission Control Protocol) / IP (Internet Protocol) protocol overhead, and bandwidth resource competition.

[0003] Currently, virtual memory file systems based on CXL (Compute Express Link, an industry-standard interconnect technology) shared memory can achieve memory sharing among multiple hosts. However, their implementation is still in the prototype stage. For example, they only support memory allocation but not memory release, resulting in relatively simple memory allocation management. Furthermore, they do not support synchronized memory reads and writes across multiple hosts, which can lead to data inconsistencies when multiple hosts access shared memory. Summary of the Invention

[0004] This application provides a cross-host memory sharing method, system, device, and medium that can ensure that when one node writes to memory, other nodes will not write to the same memory simultaneously, thereby ensuring data consistency when multiple hosts access shared memory. The specific solution is as follows:

[0005] This application provides a cross-host memory sharing method. The memory sharing system includes a master node and several slave nodes. The memory sharing method is applied to the master node and includes:

[0006] Receive a memory application request from a target slave node; the target slave node is any one of the multiple slave nodes;

[0007] Determine the memory size requirement corresponding to the memory application request;

[0008] Matching target memory corresponding to the memory size requirement in a number of target shared memory regions;

[0009] The memory information corresponding to the target memory is returned to the target slave node so that the target slave node can determine the memory type of the target memory based on the memory information. If the memory request process of the target slave node executes a preset memory mapping method, the target memory is mapped to the virtual address space of the memory request process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory request process can use the target memory.

[0010] The present application also provides a cross-host memory sharing system, comprising a master node and several slave nodes;

[0011] The master node is configured to receive a memory application request from any slave node, determine a memory size requirement corresponding to the memory application request, match target memory corresponding to the memory size requirement in a plurality of target shared memory areas, and return memory information corresponding to the target memory to any slave node;

[0012] The slave node is used to determine the memory type of the target memory based on the memory information when it obtains the memory information returned by the master node, and after its own memory demand process executes the preset memory mapping method, it maps the target memory to the virtual address space of the memory demand process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory demand process can use the target memory.

[0013] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of the aforementioned cross-host memory sharing method when executing the computer program.

[0014] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the aforementioned cross-host memory sharing method are implemented.

[0015] The present application also provides a computer program product, including a computer program, which implements the steps of the aforementioned cross-host memory sharing method when executed by a processor.

[0016] It can be seen that through the method of the present application, if the master node in the memory sharing system receives a memory application request from any slave node, it can apply for the target memory in the target memory shared area according to the memory size requirement corresponding to the request, and feed back the memory information of the target memory to the slave node that sent the request, so that the slave node can judge the corresponding memory type according to the memory information, and if the process corresponding to the slave node executes the memory mapping method, it can map the target memory to the virtual address space in progress according to the offset corresponding to the target memory, so that the process can use the allocated target memory. In this way, the host in the memory sharing system can be divided into a master node and a slave node. When the slave node applies for memory, it initiates a memory application request to the master node. The master node performs unified resource management on the entire shared memory and responds to the memory application request of the slave node. At the same time, it provides a corresponding synchronization mechanism to ensure data consistency when multiple hosts access the shared memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A flow chart of a cross-host memory sharing method provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of shared memory partitioning provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of shared memory management provided in an embodiment of the present application;

[0021] Figure 4 A flow chart of a specific cross-host memory sharing method provided in an embodiment of the present application;

[0022] Figure 5 A schematic diagram of sorting free memory by address provided in an embodiment of the present application;

[0023] Figure 6 An interactive schematic diagram of a cross-host memory sharing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0025] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0026] In related technologies, the virtual memory file system based on CXL shared memory can realize memory sharing among multiple hosts, but its implementation is relatively in the prototype stage. For example, it only supports memory allocation but not memory release, the memory allocation management is relatively simple, and it does not support memory read and write synchronization among multiple hosts, which causes data inconsistency when multiple hosts access shared memory.

[0027] In order to overcome the above technical problems, the present application discloses a cross-host memory sharing method, system, device and medium, which can ensure that when one node writes to the memory, other nodes will not write to the memory synchronously, thereby ensuring data consistency when multiple hosts access the shared memory.

[0028] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0029] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the cross-host memory sharing method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0030] See also Figure 1 As shown, an embodiment of the present application provides a cross-host memory sharing method. The memory sharing system includes a master node and several slave nodes. The memory sharing method is applied to the master node. The method is described in detail in conjunction with the execution flow of the cross-host memory sharing method. The method includes:

[0031] Step S11: receiving a memory application request from a target slave node; the target slave node is any one of the multiple slave nodes.

[0032] In this embodiment, the hosts in the memory sharing system cluster can access the same CXL shared memory, and the hosts in the cluster can also be connected to each other through the network. The cluster is divided into Master nodes and Slave nodes. The Master node is responsible for the management of the CXL shared memory pool, including memory allocation, memory release, and memory defragmentation. It should be noted that the memory sharing system is a multi-host memory sharing system based on the CXL protocol. It can achieve unified memory addressing across nodes through direct interconnection at the physical layer. The system establishes a shared memory pool through the CXL Type 3 memory expansion device, and each computing node can directly access the remote memory space using load / store instructions.

[0033] Before allocating memory to a slave node, in order to be able to quickly find a suitable free memory area based on the memory request size of the slave node, it is necessary to store the continuous free memory in the shared memory in different areas according to size. Specifically, it is necessary to divide the shared memory area according to the preset area division parameters to obtain several divided memory areas; determine the current memory request frequency, and set the memory division parameters based on the memory request frequency; divide the several divided memory areas according to the memory division parameters and the preset memory page pairs, so as to divide the several divided memory areas into corresponding memory blocks. For example, if the preset area division parameter is 11, the memory needs to be divided into 11 span areas, and the memory division parameter N can be customized. If the memory request is frequent and there are many small memories requested, the N value can be set to a smaller value, otherwise the N value can be set to a larger value. If the preset memory page is set to 2M and the memory segmentation parameter N is set to 5, the size of each span memory area is (0~2^0)*N, (2^0~2^1)*N, (2^1~2^2)*N, up to (2^9~2^10)*N, and the memory area size of the last span memory is greater than 2^10*N, that is, the memory size stored in each span memory area is 0 to 10M, 10M to 20M, 20M to 40M, 40M-80M, 80M-160M, 160M-320M, 320M-640M, 640M-1280M, 1280M-2560M, 2560M-5120M, 5120M-10240M, and greater than 10240M. And the divided shared memory is as follows Figure 2 As shown, they are all areas corresponding to the preset memory page size, but different span areas such as Figure 3As shown. Furthermore, the memory area needs to be further divided according to the memory segmentation parameters and the preset memory pages to divide the memory area into corresponding memory blocks, namely, regions, and each span memory area is connected to multiple regions with a linked list, and each region describes a memory block. In another step, it is necessary to sort the memory blocks corresponding to the several segmented memory areas according to the preset size order, that is, from small to large, and determine the memory block information corresponding to the several memory blocks respectively; store the memory block information in the corresponding several segmented memory areas to obtain several target shared memory areas; the memory block information is the number information and memory size information of the memory blocks, that is, how many regions and the total memory size are stored on the span memory area.

[0034] Furthermore, the main node in the memory sharing system, also known as the Master node, can receive memory application requests from any slave node in the system, also known as the Slave node.

[0035] Step S12: Determine the memory size requirement corresponding to the memory application request.

[0036] In this embodiment, if the master node receives a memory request from a target slave node, it needs to parse the memory request to determine the memory size requirement corresponding to the memory request, so as to match the determined memory size requirement with the target shared memory area.

[0037] Step S13: Match target memories corresponding to the memory size requirements in several target shared memory areas.

[0038] In this embodiment, it is necessary to match the corresponding shared memory for the slave node based on the received memory request. Specifically, if the master node receives a memory request from the target slave node, it is necessary to parse the memory request to determine the memory size requirement corresponding to the memory request. Then, it is necessary to match the memory size requirement obtained by parsing with several target shared memory areas, that is, to match the memory size requirement with the span area, to determine the target memory area that meets the memory size requirement, and to determine whether there is free memory that meets the memory size requirement in the target memory area. If so, it is necessary to split the target memory that meets the memory size requirement from the free memory according to the preset memory page. It should be noted that the purpose of splitting the free memory according to the preset memory page is to meet the page alignment requirement, thereby ensuring that the starting address of each new memory block obtained after the split is an integer multiple of the page size, reducing memory fragmentation and ensuring memory utilization. On the other hand, if there is no free memory that meets the memory size requirement in the target memory area, it is necessary to find the target memory that meets the memory size requirement in the adjacent area of ​​the target memory area, that is, the next area or the previous area of ​​the span area. In this way, splitting the memory area through page alignment can reduce memory fragmentation and ensure memory utilization. In addition, through Span classification, the memory pool can be directly located to find the appropriate memory area and ensure response efficiency.

[0039] Furthermore, since setting fixed preset memory pages may more easily lead to memory fragmentation, thereby increasing the need to insert memory blocks into the span area, dynamic memory pages can be set. For example, small base pages and large memory pages can be set to implement mixed page management. For example, a 4KB base page and a 2MB large page can be set. If the released area is larger than 2MB and aligned, it will be merged into the memory block corresponding to the large page first. In this way, by setting dynamic page size allocation, the performance of large-block memory operations can be significantly improved while maintaining the efficiency of small memory allocation. In addition, small page allocation can reduce resource waste, and large page allocation can improve the efficiency of TLB (Translation Lookaside Buffer).

[0040] Step S14: Return the memory information corresponding to the target memory to the target slave node so that the target slave node can determine the memory type of the target memory based on the memory information. If the memory demand process of the target slave node executes a preset memory mapping method, the target memory is mapped to the virtual address space of the memory demand process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory demand process can use the target memory.

[0041] In this embodiment, after finding a suitable target memory, the master node needs to feed back the memory information corresponding to the target memory to the target slave node, so that the target slave node can determine the corresponding memory type based on the memory information. Specifically, the offset corresponding to the target memory needs to be determined, and the offset and the number of shared memory areas corresponding to the target memory are used as the memory information corresponding to the target memory. The memory information is then returned to the target slave node, so that the target slave node can determine whether the target memory corresponds to a single shared memory area or multiple shared memory areas based on the memory information. In other words, since the memory requested by the slave node may correspond to multiple memory areas, the offset of the memory area corresponding to the memory requested by the slave node and the corresponding number of shared memory areas need to be fed back to the slave node as the memory information of the target memory. Based on the received memory information, the slave node uses the mmap method (memory map) to map the memory area to the virtual address space of the process, thereby allowing the process to use the target memory.

[0042] Furthermore, memory mapping is divided into two cases, the first case is the mapping of a single shared memory area, and the second case is the mapping of multiple shared memory areas. In the first case, if the memory type indicates that the target memory corresponds to a single shared memory area, that is, there is only one mapped memory area, then after the memory demand executes the preset memory mapping method, that is, after the process corresponding to the slave node executes the mmap method, the physical address of the target memory in the target shared memory area can be determined based on the offset corresponding to the target memory. It should be noted that the process can determine the physical address of the target memory in the target shared memory area based on the offset in the memory information received from the slave node, and then map the target memory to the virtual address space of the memory demand process corresponding to the target slave node through the physical address, so that the memory demand process can use the target memory.

[0043] In the second case, if the memory type characterizes that the target memory corresponds to multiple shared memory areas, a virtual file needs to be created to map multiple discontinuous memory addresses to the continuous virtual address space of the process. Specifically, the target file needs to be created in the virtual file system, and the memory information received from the node is saved as a page fault offset in the private space of the target file. It should be noted that the preservation of memory information requires the mapped memory information to be sent to the kernel through an interface. After receiving the command, the kernel saves the memory information to the private space of the target file. Furthermore, if the memory-requiring process executes a preset memory mapping method, the page fault offset in the private space is mapped to the virtual address space of the memory-requiring process, and the physical address corresponding to the target memory is found through the page fault offset, so that the memory-requiring process can use the target memory based on the physical address, thereby realizing page fault mapping. In this way, it is possible to avoid mapping all memory in advance and only establish mapping when actually accessed, thereby reducing initialization overhead.

[0044] As can be seen, in this embodiment, on the one hand, the hosts in the memory sharing system can be divided into master nodes and slave nodes. When a slave node applies for memory, it initiates a memory request to the master node. The master node performs unified resource management for the entire shared memory and responds to the memory request of the slave node. At the same time, a corresponding synchronization mechanism is provided to ensure data consistency when multiple hosts access the shared memory. On the other hand, the shared memory can be divided into memory areas of different sizes and sorted according to the size of the memory areas. When a memory of a specified size is requested, the minimum memory block that can meet the memory size is quickly found in the corresponding memory area. After the memory block that meets the requirement is obtained, the remaining memory blocks are added to the corresponding memory area again. This can reduce memory fragmentation, ensure memory utilization, and directly locate the memory pool corresponding to the requirement to find a suitable memory area to ensure response efficiency. On the other hand, when there are multiple memory areas to be mapped, multiple discontinuous physical memory blocks can be mapped to the continuous virtual memory address space of the process based on the virtual file system, and the multiple physical memory blocks can be sent to the kernel to be saved in the private data of the file. Then, after the process initiates a call, the virtual address is mapped to the physical address according to the address range in the private data of the file, so that the process uses the allocated memory area.

[0045] See also Figure 4 As shown, an embodiment of the present application provides a cross-host memory sharing method. The memory sharing system includes a master node and several slave nodes. The memory sharing method is applied to the master node and includes:

[0046] Step S21: Receive a memory application request from a target slave node; the target slave node is any one of the multiple slave nodes.

[0047] Step S22: Determine the memory size requirement corresponding to the memory application request.

[0048] Step S23: Matching is performed with the plurality of target shared memory areas based on the memory size requirement to determine a target memory area that matches the memory size requirement among the plurality of target shared memory areas.

[0049] In this embodiment, the memory size requirement needs to be matched with several target shared memory regions in the shared memory, and then the target memory region that matches the memory size requirement is found from among the target shared memory regions. For example, if the memory size requirement obtained by parsing the memory application request is 12KB and the preset page size is 4KB, it can be determined that the span region to be matched is 16KB. Therefore, a 16KB shared memory region can be found among the target shared memory regions as the target memory region that matches the memory size requirement.

[0050] Step S24: determine whether there is free memory that meets the memory size requirement in the target memory area; if there is free memory that meets the memory size requirement, split the target memory that meets the memory size requirement from the free memory based on the preset memory page.

[0051] In this embodiment, it is necessary to determine whether there is free memory that meets the memory size requirement in the target memory area. Taking the content in step S23 as an example, it is necessary to extract the target memory block that meets the memory size requirement from the target memory area, and perform page-aligned splitting of the target memory block through the preset memory page to obtain a number of split memories. Specifically, it is necessary to first obtain the smallest memory block in the target memory area, that is, to obtain the smallest Region, so as to give priority to using the smallest free memory block, thereby avoiding the generation of memory fragments, and then split the target memory area according to the preset memory page, for example, splitting the 16KB area according to the 4KB page size. Since the actual applied area is 12KB, 12KB of memory can be allocated, and the remaining 4KB of memory can be inserted into the current target memory area, thereby avoiding memory waste. It can be seen that when there is free memory that meets the memory size in the target memory area, the corresponding free memory can be directly split out.

[0052] Step S25: If there is no free memory that meets the memory size requirement in the target memory area, determine a target memory that meets the memory size requirement based on adjacent areas of the target memory area.

[0053] In this embodiment, if there is no free memory in the target memory area that meets the memory size requirement, it is necessary to determine a target memory in an adjacent area that meets the memory size requirement. Specifically, the next memory area adjacent to the target memory area is first used as the current target memory area, and the process jumps to the step of determining whether there is free memory in the target memory area that meets the memory size requirement to obtain the target memory. That is, as described in steps S23 and S24, if the free memory in the 16KB target memory area cannot meet the 12KB memory size requirement, it is first necessary to search for free memory in a larger span area in ascending order. Therefore, it is first necessary to use the next memory area adjacent to the target memory area as the current target memory area to search for free memory. Similarly to the steps in step S23, it is necessary to select the smallest region in the larger span, split it as needed, and then insert the remaining memory into the corresponding span area. If the 12KB memory requirement is still not met after the ascending traversal is completed, it is necessary to traverse in descending order, using the previous memory area adjacent to the target memory area as the current target memory area to search for free memory until the 12KB memory requirement is met. In this way, it is possible to effectively avoid wasting large blocks of memory for requests with small memory requirements, thus effectively ensuring memory utilization.

[0054] For the detailed description of step S21 and step S22, please refer to the aforementioned embodiment and will not be repeated here.

[0055] It can be seen that in this embodiment, the target memory that meets the corresponding memory size requirements of the memory application request can be found in several target shared memory areas, and then the corresponding free memory can be attempted to be allocated in the target memory. If the target memory cannot meet the memory size requirements, the corresponding free memory can be matched from the adjacent memory area. In this way, the situation where requests with small memory requirements waste large blocks of memory can be effectively avoided, the memory utilization rate is effectively guaranteed, and the waste of fragmented memory is avoided.

[0056] As a preferred embodiment, in order to ensure that the shared memory area contains large continuous memory blocks, when releasing memory, it is necessary to merge the released memory with the free memory blocks before and after it. Specifically, it is necessary to record the target starting address of the memory area to be released in several target shared memory areas and the target ending address of the memory area to be released after page alignment, for example. Figure 5 As shown, all free memory blocks need to be sorted from small to large according to the address range. When releasing memory, the target starting address can be recorded as A1, and the target ending address can be recorded as A2 after page alignment. Then, the idle status of the page corresponding to the target starting address can be detected through a preset bitmap. If it is idle, the end address with the same address as the target starting address is searched, and the first memory area to be merged corresponding to the end address is merged with the memory area to be released to obtain a first merged area. For example, the bitmap can be used to determine whether the page before the memory starting address is idle. If it is idle, the binary search algorithm is used to quickly find the region with the end address A1 in the free memory linked list and merge the two regions. On the other hand, the idle status of the page corresponding to the target ending address needs to be detected through a preset bitmap. If it is idle, the start address with the same address as the target ending address is searched, and the second memory area to be merged corresponding to the start address is merged with the memory area to be released to obtain a second merged area. For example, the bitmap can be used to determine whether the page after the memory ending address is idle. If it is idle, the binary search algorithm is used to quickly find the region with the start address A2 in the free memory linked list and merge the two regions. Figure 5 Taking the case in as an example, the target starting address of the memory area to be released is 3001, and the target ending address is 3400. Since there is free memory from 600 to 3000 in the free list, these two regions are merged to form a region merged area from 600 to 3400.

[0057] As a preferred embodiment, in order to avoid data inconsistencies caused by multiple slave nodes simultaneously requesting or releasing memory, a synchronization mechanism can be provided to prevent this from happening. Specifically, if the target memory is currently occupied and the master node receives a target memory request from a memory call slave node among several slave nodes, the target memory can be set to an unavailable state through a preset interface, and the master node can provide feedback on the memory inaccessibility to the memory call slave node; the memory call slave node is a different slave node from the target slave node. For example, if slave node A initiates a memory request, the master node allocates the corresponding free memory to slave node A. At this point, slave node A can read and write to the memory normally, but the memory needs to be set to an unavailable state. If another slave node, such as slave node B, also initiates a memory request of the same size, the master node can provide feedback to slave node B indicating that the memory is currently unavailable. Furthermore, if the target memory has been released, the target memory is set to an available state through a preset interface; if the master node receives a target memory request from a memory call slave node again, the target memory is allocated to the memory call slave node. That is, if slave node A has finished reading and writing the memory, the memory can be set to available state. If slave node B initiates a memory request again, the memory can be allocated to slave node B. This ensures that when a node is writing memory, other nodes will not read and write synchronously, which will cause data inconsistency.

[0058] like Figure 6 The figure shows an interactive diagram of the cross-host memory sharing method of the present application, in which a slave node (slave node) needs to initiate a memory request to the master node (master node). The master node then needs to allocate the corresponding target memory in the shared memory for the slave node based on the memory request and feedback the memory information corresponding to the target memory, including the memory address, to the slave node so that the slave node can access the shared memory. In addition, for the slave node to access the shared memory, it needs to execute the mmap memory mapping method from the memory request process corresponding to the node to map the target memory to the virtual address space of the memory request process based on the offset corresponding to the target memory in the memory information, thereby allowing the memory request process to use the target memory.

[0059] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0060] An embodiment of the present application further provides a cross-host memory sharing system, comprising a master node and a plurality of slave nodes;

[0061] The master node is configured to receive a memory application request from any slave node, determine a memory size requirement corresponding to the memory application request, match target memory corresponding to the memory size requirement in a plurality of target shared memory areas, and return memory information corresponding to the target memory to any slave node;

[0062] The slave node is used to determine the memory type of the target memory based on the memory information when it obtains the memory information returned by the master node. If its own memory-demanding process executes the preset memory mapping method, the target memory is mapped to the virtual address space of the memory-demanding process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory-demanding process can use the target memory.

[0063] In this embodiment, if the master node in the memory sharing system receives a memory application request from any slave node, it can apply for target memory in the target memory shared area according to the memory size requirement corresponding to the request, and feed back the memory information of the target memory to the slave node that sent the request, so that the slave node can determine the corresponding memory type according to the memory information, and if the process corresponding to the slave node executes the memory mapping method, it can map the target memory to the virtual address space in progress according to the offset corresponding to the target memory, so that the process can use the allocated target memory. In this way, the host in the memory sharing system can be divided into master nodes and slave nodes. When applying for memory, the slave node initiates a memory application request to the master node. The master node performs unified resource management on the entire shared memory and responds to the memory application request of the slave node. At the same time, a corresponding synchronization mechanism is provided to ensure data consistency when multiple hosts access the shared memory.

[0064] In some embodiments, the cross-host memory sharing system is further configured to:

[0065] The shared memory area is divided according to the preset area division parameters to obtain a plurality of divided memory areas;

[0066] Determine the current memory request frequency and set memory segmentation parameters based on the memory request frequency;

[0067] The plurality of divided memory areas are divided respectively according to the memory division parameter and the preset memory page pairs, so as to divide the plurality of divided memory areas into corresponding plurality of memory blocks.

[0068] In some embodiments, the cross-host memory sharing system is further configured to:

[0069] sorting the memory blocks corresponding to the divided memory areas based on a preset size order, and determining the memory block information corresponding to the memory blocks;

[0070] The memory block information is stored in the corresponding divided memory areas to obtain the target shared memory areas; the memory block information includes the number of memory blocks and the memory size information.

[0071] In some embodiments, the master node is specifically used to:

[0072] If the master node receives a memory request from the target slave node, it parses the memory request to determine the memory size requirement corresponding to the memory request;

[0073] Matching the target shared memory regions with the target shared memory regions based on the memory size requirement to determine a target memory region matching the memory size requirement among the target shared memory regions;

[0074] Determine whether there is free memory that meets the memory size requirement in the target memory area, and if there is free memory that meets the memory size requirement, split the target memory that meets the memory size requirement from the free memory based on the preset memory pages;

[0075] If there is no free memory that meets the memory size requirement in the target memory area, a target memory that meets the memory size requirement is determined based on adjacent areas of the target memory area.

[0076] In some embodiments, the master node is specifically used to:

[0077] Extracting a target memory block that meets the memory size requirement from the target memory area, and performing page-aligned splitting on the target memory block using preset memory pages to obtain a plurality of split memories;

[0078] Target free memory that meets the memory size requirement is extracted from the plurality of split memories, and the target free memory is used as the target memory.

[0079] In some embodiments, the master node is specifically used to:

[0080] If there is no free memory that meets the memory size requirement in the target memory area, then the next memory area adjacent to the target memory area is used as the current target memory area, and the process jumps to the step of determining whether there is free memory that meets the memory size requirement in the target memory area to obtain the target memory;

[0081] If the target memory is not obtained, the previous memory area adjacent to the target memory area is used as the current target memory area, and the process jumps to the step of determining whether there is free memory that meets the memory size requirement in the target memory area to obtain the target memory.

[0082] In some embodiments, the master node is specifically used to:

[0083] An offset corresponding to the target memory is determined, and the offset and the number of shared memory regions corresponding to the target memory are used as memory information corresponding to the target memory. The memory information is returned to the target slave node, so that the target slave node determines whether the target memory corresponds to a single shared memory region or multiple shared memory regions based on the memory information.

[0084] In some embodiments, the slave node is specifically configured to:

[0085] If the memory type indicates that the target memory corresponds to a single shared memory region, and the memory request process executes a preset memory mapping method, determining a physical address of the target memory in the target shared memory region based on an offset corresponding to the target memory;

[0086] The target memory is mapped to the virtual address space of the target slave node corresponding to the memory-requiring process through the physical address, so that the memory-requiring process can use the target memory.

[0087] In some embodiments, the slave node is specifically configured to:

[0088] If the memory type indicates that the target memory corresponds to multiple shared memory areas, a target file is created and the memory information is saved as a fault page offset in the private space of the target file;

[0089] If the memory-requiring process executes a preset memory mapping method, the page fault offset in the private space is mapped to the virtual address space of the memory-requiring process, and the physical address corresponding to the target memory is found through the page fault offset, so that the memory-requiring process can use the target memory based on the physical address.

[0090] In some embodiments, the master node is also used to:

[0091] Record the target start address of the memory area to be released in several target shared memory areas and the target end address of the memory area to be released after page alignment;

[0092] Detecting the idle state of the page corresponding to the target start address through a preset bitmap, and if the page is idle, searching for the end address that is the same as the target start address, and merging the first memory area to be merged corresponding to the end address with the memory area to be released to obtain a first merged area;

[0093] And / or, the idle state of the page corresponding to the target end address is detected through a preset bitmap. If it is idle, the starting address identical to the target end address is searched, and the second memory area to be merged corresponding to the starting address is merged with the memory area to be released to obtain a second merged area.

[0094] In some embodiments, the master node is also used to:

[0095] If the target memory is currently occupied and a target memory application request for obtaining the target memory is received from a memory calling slave node among several slave nodes, the target memory is set to an unavailable state through a preset interface, and memory inaccessibility information is fed back to the memory calling slave node; the memory calling slave node is a slave node different from the target slave node.

[0096] In some embodiments, the master node is also used to:

[0097] If the target memory has been released, the target memory is set to be available through the preset interface;

[0098] If the master node receives the target memory application request sent by the memory calling slave node again, the master node allocates the target memory to the memory calling slave node.

[0099] For descriptions of features in the embodiments corresponding to the cross-host memory sharing system, reference may be made to the relevant descriptions of the embodiments corresponding to the cross-host memory sharing method, which will not be detailed here.

[0100] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned cross-host memory sharing method embodiments.

[0101] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned cross-host memory sharing method embodiments when running.

[0102] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0103] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the computer program implements the steps of any of the above-mentioned cross-host memory sharing method embodiments.

[0104] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps of any of the above-mentioned cross-host memory sharing method embodiments.

[0105] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0106] The above is a detailed introduction to the cross-host memory sharing method, system, device and medium provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A cross-host memory sharing method, characterized in that: The memory sharing system includes a master node and several slave nodes. The memory sharing method is applied to the master node, including: receiving a memory application request from a target slave node; the target slave node being any one of the plurality of slave nodes; Determine the memory size requirement corresponding to the memory application request; matching target memories corresponding to the memory size requirements in a plurality of target shared memory areas; Returning memory information corresponding to the target memory to the target slave node, so that the target slave node determines the memory type of the target memory based on the memory information; if the memory request process of the target slave node executes a preset memory mapping method, mapping the target memory to the virtual address space of the memory request process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory request process can use the target memory; Before receiving the memory application request from the target slave node, the method further includes: sorting the memory blocks corresponding to the divided memory areas based on a preset size order, and determining the memory block information corresponding to the memory blocks; Storing the memory block information in the corresponding plurality of divided memory areas to obtain a plurality of target shared memory areas; the memory block information includes the number of memory blocks and memory size information; wherein the plurality of divided memory areas are memory areas obtained by dividing a shared memory area according to preset area division parameters; and the plurality of memory blocks are memory blocks obtained by dividing the plurality of divided memory areas respectively; The step of returning the memory information corresponding to the target memory to the target slave node so that the target slave node determines the memory type of the target memory based on the memory information includes: Determine the offset corresponding to the target memory, and use the offset and the number of shared memory areas corresponding to the target memory as memory information corresponding to the target memory, and return the memory information to the target slave node so that the target slave node determines that the target memory corresponds to a single shared memory area or multiple shared memory areas based on the memory information.

2. The cross-host memory sharing method according to claim 1, characterized in that: Before receiving the memory application request from the target slave node, the method further includes: The shared memory area is divided according to the preset area division parameters to obtain a plurality of divided memory areas; Determining a current memory request frequency, and setting a memory segmentation parameter based on the memory request frequency; The plurality of divided memory areas are divided respectively according to the memory division parameters and the preset memory page pairs, so as to divide the plurality of divided memory areas into corresponding plurality of memory blocks.

3. The cross-host memory sharing method according to claim 2, characterized in that: The matching of target memories corresponding to the memory size requirement in the plurality of target shared memory areas includes: Matching the memory size requirement with the plurality of target shared memory regions to determine a target memory region matching the memory size requirement among the plurality of target shared memory regions; Determine whether there is free memory that meets the memory size requirement in the target memory area, and if there is free memory that meets the memory size requirement, split the target memory that meets the memory size requirement from the free memory based on the preset memory page; If there is no free memory that meets the memory size requirement in the target memory area, a target memory that meets the memory size requirement is determined based on adjacent areas of the target memory area.

4. The cross-host memory sharing method according to claim 3, characterized in that: The step of splitting the target memory that meets the memory size requirement from the free memory based on the preset memory page includes: Extracting a target memory block that meets the memory size requirement from the target memory area, and performing page-aligned splitting on the target memory block using the preset memory pages to obtain a plurality of split memories; A target free memory that meets the memory size requirement is extracted from the plurality of split memories, and the target free memory is used as the target memory.

5. The cross-host memory sharing method according to claim 3, characterized in that: If there is no free memory that meets the memory size requirement in the target memory area, determining a target memory that meets the memory size requirement based on an adjacent area of ​​the target memory area includes: If there is no free memory that meets the memory size requirement in the target memory area, taking the next memory area adjacent to the target memory area as the current target memory area, and jumping to the step of determining whether there is free memory that meets the memory size requirement in the target memory area to obtain the target memory; If the target memory is not obtained, the previous memory area adjacent to the target memory area is used as the current target memory area, and the process jumps to the step of determining whether there is free memory in the target memory area that meets the memory size requirement to obtain the target memory.

6. The cross-host memory sharing method according to claim 1, characterized in that: If the memory request process of the target slave node executes a preset memory mapping method, mapping the target memory to the virtual address space of the memory request process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory request process uses the target memory, including: If the memory type indicates that the target memory corresponds to a single shared memory area, and the memory request process executes a preset memory mapping method, determining a physical address of the target memory in the target shared memory area based on an offset corresponding to the target memory; The target memory is mapped to the virtual address space of the target slave node corresponding to the memory-requiring process through the physical address, so that the memory-requiring process can use the target memory.

7. The cross-host memory sharing method according to claim 1, characterized in that: If the memory request process of the target slave node executes a preset memory mapping method, mapping the target memory to the virtual address space of the memory request process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory request process uses the target memory, including: If the memory type indicates that the target memory corresponds to multiple shared memory areas, a target file is created, and the memory information is saved as a fault page offset in a private space of the target file; If the memory-requiring process executes the preset memory mapping method, the page fault offset in the private space is mapped to the virtual address space of the memory-requiring process, and the physical address corresponding to the target memory is found through the page fault offset, so that the memory-requiring process can use the target memory based on the physical address.

8. The cross-host memory sharing method according to claim 1, characterized in that: Also includes: Record the target starting addresses of the memory areas to be released in the plurality of target shared memory areas and the target ending addresses of the memory areas to be released after page alignment; Detecting the idle state of the page corresponding to the target start address through a preset bitmap, and if the page is idle, searching for an end address that is the same as the target start address, and merging the first to-be-merged memory area corresponding to the end address with the to-be-released memory area to obtain a first merged area; And / or, the idle state of the page corresponding to the target end address is detected through a preset bitmap. If it is idle, the starting address identical to the target end address is searched, and the second memory area to be merged corresponding to the starting address is merged with the memory area to be released to obtain a second merged area.

9. The cross-host memory sharing method according to any one of claims 1 to 8, characterized in that: Also includes: If the target memory is currently occupied and a target memory application request for obtaining the target memory is received from a memory calling slave node among the plurality of slave nodes, the target memory is set to an unavailable state through a preset interface, and memory inaccessibility information is fed back to the memory calling slave node; The memory calling slave node is a slave node different from the target slave node.

10. The cross-host memory sharing method according to claim 9, characterized in that: Also includes: If the target memory has been released, setting the target memory to an available state through the preset interface; If the target memory application request sent by the memory calling slave node is received again, the target memory is allocated to the memory calling slave node.

11. A cross-host memory sharing system, characterized in that: Includes a master node and several slave nodes; The master node is configured to receive a memory application request from any slave node, determine a memory size requirement corresponding to the memory application request, match a target memory corresponding to the memory size requirement in a plurality of target shared memory areas, and return memory information corresponding to the target memory to any slave node; The slave node is configured to, upon obtaining the memory information returned by the master node, determine the memory type of the target memory based on the memory information; and if its own memory-requiring process executes a preset memory mapping method, map the target memory to the virtual address space of the memory-requiring process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory-requiring process can use the target memory; The cross-host memory sharing system is used to: sorting the memory blocks corresponding to the divided memory areas based on a preset size order, and determining the memory block information corresponding to the memory blocks; Storing the memory block information in the corresponding plurality of divided memory areas to obtain a plurality of target shared memory areas; the memory block information includes the number of memory blocks and memory size information; wherein the plurality of divided memory areas are memory areas obtained by dividing a shared memory area according to preset area division parameters; and the plurality of memory blocks are memory blocks obtained by dividing the plurality of divided memory areas respectively; The master node is used to: An offset corresponding to the target memory is determined, and the offset and the number of shared memory regions corresponding to the target memory are used as memory information corresponding to the target memory. The memory information is returned to the target slave node, so that the target slave node determines whether the target memory corresponds to a single shared memory region or multiple shared memory regions based on the memory information.

12. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the cross-host memory sharing method as claimed in any one of claims 1 to 10 when executing the computer program.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the cross-host memory sharing method according to any one of claims 1 to 10 are implemented.

Images