Cross-host memory sharing method, system, equipment and medium

Abstract

The invention discloses a cross-host memory sharing method, system and device and a medium, the memory sharing system comprises a master node and a plurality of slave nodes, the memory sharing method is applied to the master node, and the method comprises the steps that a memory application request of a target slave node is received; applying for a target memory in a target memory sharing area according to a memory size requirement corresponding to the request, feeding back memory information of the target memory to a target slave node, so that the target slave node judges a corresponding memory type according to the memory information, and if a process corresponding to the target slave node executes a memory mapping method, sending the target memory to the target slave node. If yes, the target memory can be mapped to the proceeding virtual address space according to the target type and the offset corresponding to the target memory, so that the process can use the distributed target memory. Therefore, it can be guaranteed that when one node writes the memory, other nodes cannot synchronously write the memory, and then the data consistency of the shared memory accessed by multiple hosts is guaranteed.

Description

Technical Field

[0001] This application relates to the technical field of memory sharing, and in particular, to a method, system, device, and medium for cross-host memory sharing. Background Art

[0002] In artificial intelligence training, big data analysis, and cloud computing platforms, data interaction between multi-host clusters is becoming increasingly frequent. Although traditional network transmission solutions based on RDMA (Remote Direct Memory Access) reduce some latency by bypassing the operating system protocol stack, they are still limited by problems such as physical network topology, TCP (Transmission Control Protocol) / IP (Internet Protocol) protocol overhead, and bandwidth resource competition.

[0003] Currently, a virtual memory file system based on CXL (Compute Express Link, an industry-standard interconnection technology) shared memory can achieve memory sharing between multiple hosts. However, its implementation is relatively in the prototype stage. For example, it only supports memory allocation but not memory release, the memory allocation management is relatively simple, and it does not support memory read / write synchronization between multiple hosts, thus causing data inconsistency problems when multiple hosts access shared memory. Summary of the Invention

[0004] This application provides a method, system, device, and medium for cross-host memory sharing, which can ensure that when one node writes to memory, other nodes do not synchronously write to the same memory, thereby ensuring data consistency when multiple hosts access shared memory. The specific solutions are as follows: This application provides a method for cross-host memory sharing. The memory sharing system includes a primary node and several secondary nodes. The memory sharing method is applied to the primary node and includes: Receiving a memory application request from a target secondary node; the target secondary node is any one of the several secondary nodes; Determining the memory size requirement corresponding to the memory application request; Matching a target memory corresponding to the memory size requirement in several target shared memory regions; Returning the memory information corresponding to the target memory to the target secondary node, so that the target secondary node can determine the memory type of the target memory based on the memory information. If the memory requirement process of the target secondary node executes a preset memory mapping method, the target memory is mapped to the virtual address space of the memory requirement process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory requirement process can use the target memory.

[0005] The present application also provides a cross-host memory sharing system, including a master node and several slave nodes; The master node is configured to receive a memory application request from any slave node and determine the memory size requirement corresponding to the memory application request; match a target memory corresponding to the memory size requirement in several target shared memory areas; and return the memory information corresponding to the target memory to any slave node; The slave node is configured to, when obtaining the memory information returned by the master node, determine the memory type of the target memory based on the memory information, and after the preset memory mapping method is executed by the memory requirement process of the slave node itself, map the target memory to the virtual address space of the memory requirement process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory requirement process can use the target memory.

[0006] The present application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of the aforementioned cross-host memory sharing method when executing the computer program.

[0007] The present application also provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the aforementioned cross-host memory sharing method.

[0008] The present application also provides a computer program product, including a computer program, and the computer program, when executed by a processor, implements the steps of the aforementioned cross-host memory sharing method.

[0009] Thus, through the method of the present application, if the master node in the memory sharing system receives a memory application request from any slave node, it can apply for a target memory in the target memory sharing area according to the corresponding memory size requirement of the request, and feedback the memory information of the target memory to the slave node that sends the request, so that the slave node can determine the corresponding memory type according to the memory information, and if the process corresponding to the slave node executes the memory mapping method, it can map the target memory to the virtual address space of the process according to the offset corresponding to the target memory, so that the process can use the allocated target memory. In this way, the hosts in the memory sharing system can be divided into a master node and slave nodes. When a slave node applies for memory, it sends a memory application request to the master node. The master node performs unified resource management on the entire shared memory and responds to the memory application requests of the slave nodes, and at the same time provides a corresponding synchronization mechanism to ensure data consistency when multiple hosts access the shared memory. Description of the Drawings

[0010] To more clearly illustrate the embodiments of the present application, the accompanying drawings required for the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0011] Figure 1 Flowchart of a method for cross-host memory sharing provided by an embodiment of the present application; Figure 2 Schematic diagram of shared memory partitioning provided by an embodiment of the present application; Figure 3 Schematic diagram of shared memory management provided by an embodiment of the present application; Figure 4 Flowchart of a specific method for cross-host memory sharing provided by an embodiment of the present application; Figure 5 Schematic diagram of sorting free memory by address provided by an embodiment of the present application; Figure 6 Interaction diagram of a method for cross-host memory sharing provided by an embodiment of the present application. Detailed implementation manners

[0012] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present application.

[0013] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variation thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0014] In the related art, a virtual memory file system for CXL-based shared memory can achieve memory sharing between multiple hosts. However, its implementation is relatively in the prototype stage. For example, it only supports memory allocation and does not support memory release. The memory allocation management is relatively simple, and it does not support memory read / write synchronization between multiple hosts, thus causing data inconsistency problems when multiple hosts access the shared memory.

[0015] To overcome the above technical problems, the present application discloses a cross-host memory sharing method, system, device, and medium, which can ensure that when one node writes to memory, other nodes do not synchronously write to the same memory, thereby ensuring data consistency for multiple hosts accessing shared memory.

[0016] To enable those skilled in the art of this technology to better understand the solution of the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] Combined with the specific application environment architecture or specific hardware architecture on which the execution of the cross-host memory sharing method depends, the specific application environment architecture or specific hardware architecture will be described herein.

[0018] See Figure 1 As shown, an embodiment of the present application provides a cross-host memory sharing method. The memory sharing system includes a master node and several slave nodes. The memory sharing method is applied to the master node. Combining with the execution process of the cross-host memory sharing method, the method will be described in detail. The method includes: Step S11: Receive a memory application request from a target slave node; the target slave node is any one of the several slave nodes.

[0019] In this embodiment, the hosts in the memory sharing system cluster can access the same CXL shared memory, and the hosts in the cluster can also be connected to each other through a network. The cluster is divided into a Master node and a Slave node. The Master node is responsible for the management of the CXL shared memory pool, including memory allocation, memory release, and memory fragmentation reorganization. It should be noted that the memory sharing system is a multi-host memory sharing system based on the CXL protocol, which can achieve unified memory addressing across nodes through direct physical layer interconnection, and the system establishes a shared memory pool through a CXL Type3 memory expansion device, and each computing node can directly access the remote memory space with load / store instructions.

[0020] Before allocating memory for slave nodes, in order to quickly find a suitable free memory area according to the memory application size of slave nodes, it is necessary to store the continuous free memory in the shared memory in different areas according to the size. Specifically, it is necessary to divide the shared memory area according to the preset area division parameters to obtain several divided memory areas; determine the current memory application frequency, and set the memory division parameters based on the memory application frequency; divide several divided memory areas according to the memory division parameters and the preset memory pages respectively, so as to divide several divided memory areas into corresponding memory blocks respectively. For example, if the preset area division parameter is 11, the memory needs to be divided into 11 span areas, and the memory division parameter N can be customized. If the memory application is relatively frequent and there are many small memory applications, the value of N can be set to a smaller value, otherwise the value of N can be set to a larger value. If the preset memory page is set to 2M and the memory division parameter N is set to 5, the size of each span memory area is (0~2^0)*N, (2^0~2^1)*N, (2^1~2^2)*N, up to (2^9~2^10)*N. The memory area size of the last span memory is greater than 2^10*N, that is, the memory size stored in each span memory area is 0 to 10M, 10M to 20M, 20M to 40M, 40M - 80M, 80M - 160M, 160M - 320M, 320M - 640M, 640M - 1280M, 1280M - 2560M, 2560M - 5120M, 5120M - 10240M, greater than 10240M. And the divided shared memory is as Figure 2 shown, all are areas corresponding to the preset memory page size, but different span areas are as Figure 3 shown. Further, it is necessary to further divide the memory area according to the memory division parameters and the preset memory page to divide the memory area into corresponding memory blocks, that is, regions, and each span memory area connects multiple regions with a linked list, and each region describes a memory block. Furthermore, it is necessary to sort the memory blocks corresponding to several divided memory areas according to the preset size order, that is, in ascending order, and determine the memory block information corresponding to several memory blocks respectively; store the memory block information in the corresponding several divided memory areas to obtain several target shared memory areas; the memory block information is the quantity information and memory size information of the memory block, that is, how many regions are on this span memory area and the total memory size.

[0021] Furthermore, the memory application request of any slave node, that is, the Slave node, in the system can be received through the master node in the memory sharing system, that is, the Master node.

[0022] Step S12: Determine the memory size requirement corresponding to the memory application request.

[0023] In this embodiment, if the master node receives a memory application request from a target slave node, it is necessary to parse the memory application request to determine the memory size requirement corresponding to the memory application request, so as to match the determined memory size requirement with the target shared memory area.

[0024] Step S13: Match a target memory corresponding to the memory size requirement in a number of target shared memory areas.

[0025] In this embodiment, it is necessary to match a corresponding shared memory for the slave node according to the received memory application request. Specifically, if the master node receives a memory application request from a target slave node, it is necessary to parse the memory application request to determine the memory size requirement corresponding to the memory application request. Then, it is necessary to match the parsed memory size requirement with a number of target shared memory areas, that is, to match the memory size requirement with the span area, so as to determine a target memory area that meets the memory size requirement, and determine whether there is free memory that meets the memory size requirement in the target memory area. If there is, it is necessary to split the free memory that meets the memory size requirement from the free memory according to the preset memory page. It should be noted that splitting the free memory by the preset memory page is to meet the page alignment requirement, so as to ensure that the starting address of each new memory block obtained after splitting is an integer multiple of the page size, reduce memory fragmentation, and ensure memory utilization. On the other hand, if there is no free memory that meets the memory size requirement in the target memory area, it is necessary to find a target memory that meets the memory size requirement in the adjacent area of the target memory area, that is, the next area or the previous area of the span area. In this way, splitting the memory area by page alignment can reduce memory fragmentation and ensure memory utilization, and directly locate the memory pool through Span classification to find a suitable memory area to ensure the response efficiency.

[0026] Furthermore, since setting a fixed preset memory page may more easily lead to memory fragmentation, thereby increasing the need to insert memory blocks into the span area, a dynamic memory page can be set. For example, a small base page and a large memory page can be set to implement hybrid page management. For example, a 4KB base page and a 2MB large page can be set. If the release area is greater than 2MB and aligned, it is preferentially merged into a memory block corresponding to the large page. In this way, through the set dynamic page size allocation, while maintaining the efficiency of small memory allocation, the performance of large memory operations can be significantly improved. And through small page allocation, resource waste can be reduced, and through large page allocation, the efficiency of the TLB (Translation Lookaside Buffer) can be improved.

[0027] Step S14: Return the memory information corresponding to the target memory to the target slave node, so that the target slave node can determine the memory type of the target memory based on the memory information. If the memory requirement process of the target slave node executes a preset memory mapping method, then map the target memory to the virtual address space of the memory requirement process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory requirement process can use the target memory.

[0028] In this embodiment, after finding a suitable target memory, it is necessary to feedback the memory information corresponding to the target memory to the target slave node through the master node, so that the target slave node can determine the corresponding memory type according to the memory information. Specifically, it is necessary to determine the offset corresponding to the target memory, and use the offset and the number of shared memory regions corresponding to the target memory as the memory information corresponding to the target memory, and return the memory information to the target slave node, so that the target slave node can determine whether the target memory corresponds to a single shared memory region or multiple shared memory regions based on the memory information. That is to say, since the memory applied by the slave node may correspond to multiple memory regions, it is necessary to feedback the offset of the memory region corresponding to the memory applied by the slave node and the corresponding number of shared memory regions as the memory information of the target memory to the slave node, so that the slave node can map the memory region to the virtual address space of the process through the mmap method (memory map) based on the received memory information, and then enable the process to use the target memory.

[0029] Further, memory mapping is divided into two cases. The first case is the mapping of a single shared memory region, and the second case is the mapping of multiple shared memory regions. In the first case, if the memory type indicates that the target memory corresponds to a single shared memory region, that is, there is only one mapped memory region, then after the memory requirement executes the preset memory mapping method, that is, after the process corresponding to the slave node executes the mmap method, the physical address of the target memory in the target shared memory region can be determined according to the offset corresponding to the target memory. It should be noted that the process can determine the physical address of the target memory in the target shared memory region according to the offset in the memory information received from the slave node, and then map the target memory to the virtual address space of the memory requirement process corresponding to the target slave node through the physical address, so that the memory requirement process can use the target memory.

[0030] In the second case, if the memory type characterizes that the target memory corresponds to multiple shared memory areas, a virtual file needs to be created to map multiple discontinuous memory addresses to the continuous virtual address space of the process. Specifically, the target file needs to be created in the virtual file system, and the memory information received from the node is saved as a page fault offset to the private space of the target file. It should be noted that the preservation of memory information requires the mapped memory information to be sent to the kernel through an interface. After receiving the command, the kernel saves the memory information to the private space of the target file. Further, if the memory demand process executes a preset memory mapping method, the page fault offset in the private space is mapped to the virtual address space of the memory demand process, and the physical address corresponding to the target memory is found through the page fault offset, so that the memory demand process uses the target memory based on the physical address, thereby realizing page fault mapping. In this way, it is possible to avoid mapping all memory in advance, and only establish mapping when actually accessing, thereby reducing initialization overhead.

[0031] It can be seen that in this embodiment, on the one hand, the host in the memory sharing system can be divided into a master node and a slave node. When the slave node applies for memory, it initiates a memory application request to the master node. The master node performs unified resource management on the entire shared memory and responds to the memory application request of the slave node, and provides a corresponding synchronization mechanism to ensure data consistency when multiple hosts access the shared memory; on the other hand, the shared memory can be divided into memory areas of different sizes and sorted according to the size of the memory area. When applying for memory of a specified size, the minimum memory block that can meet the memory size is quickly found in the corresponding memory area. After the memory block that meets the requirement is obtained, the remaining memory blocks are added to the corresponding memory area again, thereby reducing memory fragmentation, ensuring memory utilization, and directly locating the memory pool corresponding to the requirement, finding a suitable memory area, and ensuring response efficiency; on the other hand, when there are multiple memory areas to be mapped, multiple discontinuous physical memory blocks can be mapped to the continuous virtual memory address space of the process based on the virtual file system, and multiple physical memory blocks are sent to the kernel to save in the private data of the file, and then after the process initiates the call, the virtual address is mapped to the physical address according to the address range in the private data of the file, so that the process uses the allocated memory area.

[0032] See also Figure 4 As shown, an embodiment of the present application provides a cross-host memory sharing method, the memory sharing system includes a master node and a plurality of slave nodes, and the memory sharing method is applied to the master node, including: Step S21, receiving a memory application request from a target slave node; the target slave node is any node among a plurality of slave nodes.

[0033] Step S22: Determine the memory size requirement corresponding to the memory application request.

[0034] Step S23: Match the memory size requirement with a number of target shared memory regions to determine the target memory region in the number of target shared memory regions that matches the memory size requirement.

[0035] In this embodiment, it is necessary to match the memory size requirement with a number of target shared memory regions in the shared memory, and then find the target memory region that matches the memory size requirement from the number of target shared memory regions. For example, if the memory size requirement parsed from the memory application request is 12KB and the size of the preset page is 4KB, it can be determined that the span region to be matched should be 16KB. Therefore, a 16KB shared memory region can be searched in the number of target shared memory regions as the target memory region that matches the memory size requirement.

[0036] Step S24: Determine whether there is free memory in the target memory region that meets the memory size requirement. If there is free memory that meets the memory size requirement, split out the target memory that meets the memory size requirement from the free memory based on the preset memory page.

[0037] In this embodiment, it is necessary to determine whether there is free memory in the target memory region that meets the memory size requirement. Taking the content in step S23 as an example, it is necessary to extract the target memory block that meets the memory size requirement from the target memory region, and perform page alignment splitting on the target memory block through the preset memory page to obtain a number of split memories. Specifically, first, it is necessary to obtain the smallest memory block in the target memory region, that is, obtain the smallest Region, so as to preferentially use the smallest free memory block and thus avoid the generation of memory fragmentation. Then, the target memory region is split according to the preset memory page. For example, a 16KB region is split according to a page size of 4KB. Since the actually applied region is 12KB, 12KB of memory can be allocated, and the remaining 4KB of memory can be inserted into the current target memory region, thus avoiding waste of memory. It can be seen that when there is free memory in the target memory region that meets the memory size, the corresponding free memory can be directly split out.

[0038] Step S25: If there is no free memory in the target memory region that meets the memory size requirement, determine the target memory that meets the memory size requirement based on the adjacent region of the target memory region.

[0039] In this embodiment, if there is no free memory in the target memory area that meets the memory size requirement, it is necessary to determine the target memory that meets the memory size requirement in the adjacent area. Specifically, first, the next memory area adjacent to the target memory area needs to be used as the current target memory area, and then jump to the step of determining whether there is free memory in the target memory area that meets the memory size requirement to obtain the target memory. That is, as described in steps S23 and S24, if the free memory in the 16KB target memory area cannot meet the 12KB memory size requirement, it is first necessary to search for free memory in ascending order in larger span areas. Therefore, the next memory area adjacent to the target memory area needs to be used as the current target memory area to search for free memory. And the same as the steps in S23, it is necessary to take the smallest region in the larger span, split it as needed, and then insert the remaining memory into the corresponding span area. If the 12KB memory requirement is still not met after ascending traversal, it is necessary to perform a descending traversal, and use the previous memory area adjacent to the target memory area as the current target memory area to search for free memory until the 12KB memory requirement is met. In this way, it can effectively avoid the situation where requests for small memory requirements waste large chunks of memory, and effectively ensure the utilization rate of memory.

[0040] For the specific descriptions of steps S21 and S22, reference can be made to the foregoing embodiments, and details will not be elaborated herein.

[0041] It can be seen that in this embodiment, the target memory that meets the corresponding memory size requirement of the memory application request can be found in several target shared memory areas, and then an attempt is made to allocate the corresponding free memory in the target memory. When the target memory cannot meet the memory size requirement, the corresponding free memory is matched from the adjacent memory areas. In this way, it can effectively avoid the situation where requests for small memory requirements waste large chunks of memory, effectively ensure the utilization rate of memory, and avoid the waste of fragmented memory.

[0042] As a preferred embodiment, in order to ensure that there are large continuous memory blocks in the shared memory area, it is necessary to merge the released memory with the adjacent free memory blocks before and after it when releasing the memory. Specifically, it is necessary to record the target start address of the memory area to be released and the target end address after page alignment of the memory area to be released in several target shared memory areas. For example, Figure 5As shown, all free memory blocks need to be sorted in ascending order of address range. When releasing memory, the target start address can be recorded as A1, and the target end address after page alignment can be recorded as A2. Then, the free status of the page corresponding to the target start address can be detected through a preset bitmap. If it is free, find the end address that is the same as the target start address, and merge the first memory region to be merged corresponding to the end address with the memory region to be released to obtain the first merged region. For example, it can be determined through the bitmap whether the previous page of the memory start address is free. If it is free, the region with the end address of A1 can be quickly found in the free memory linked list through the binary search algorithm and the two regions can be merged. On the other hand, it is necessary to detect the free status of the page corresponding to the target end address through a preset bitmap. If it is free, find the start address that is the same as the target end address, and merge the second memory region to be merged corresponding to the start address with the memory region to be released to obtain the second merged region. For example, it can be determined through the bitmap whether the next page of the memory end address is free. If it is free, the region with the start address of A2 can be quickly found in the free memory linked list through the binary search algorithm and the two regions can be merged. Taking Figure 5 the situation in Figure 5 as an example, the target start address of the memory region to be released is 3001, and the target end address is 3400. Since there is free memory from 600 to 3000 in the free linked list, these two regions are merged to finally form a merged region from 600 to 3400.

[0043] As a preferred embodiment, to avoid individual slave nodes from simultaneously applying for or releasing memory, which may lead to data inconsistency problems during concurrent operations, a synchronization mechanism can be provided to avoid the occurrence of the above situation. Specifically, if the target memory is currently occupied and the master node receives a target memory application request for obtaining the target memory sent by a memory-calling slave node among several slave nodes, the target memory can be set to an unavailable state through a preset interface, and the memory-inaccessible information can be fed back to the memory-calling slave node by the master node; the memory-calling slave node is a slave node different from the target slave node. For example, if slave node A initiates a memory application, the master node allocates corresponding free memory to slave node A. At this time, slave node A can normally read and write this memory, and this memory needs to be set to an unavailable state. If other slave nodes, such as slave node B, also initiate a memory application of the same size, the master node can feed back to slave node B that this memory is currently in an unavailable state. Further, if the target memory has been released, the target memory is set to an available state through a preset interface; if the master node receives a target memory application request sent by a memory-calling slave node again, the target memory is allocated to the memory-calling slave node. That is, if slave node A finishes reading and writing this memory, this memory can be set to an available state. If slave node B initiates a memory application request again, this memory can be allocated to slave node B at this time. In this way, it can be ensured that when a node writes to memory, the other nodes will not synchronously read and write, thus avoiding the problem of data inconsistency.

[0044] As Figure 6 shown, it is an interaction schematic diagram of the cross-host memory sharing method of this application. Among them, the Slave node (slave node) needs to initiate a memory application request to the Master node (master node). Then, the Master node needs to allocate corresponding target memory for the Slave node in the shared memory according to the memory application request, and feed back the memory information including the memory address corresponding to the target memory to the Slave node so that the Slave node can access the shared memory. And for the Slave node to access the shared memory, the memory demand process corresponding to the slave node needs to execute the mmap memory mapping method to map the target memory to the virtual address space of the memory demand process according to the offset corresponding to the target memory in the memory information, so that the memory demand process can use the target memory.

[0045] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0046] The embodiment of this application also provides a cross-host memory sharing system, including a master node and several slave nodes; The master node is used to receive a memory application request from any slave node, and determine the memory size requirement corresponding to the memory application request; match a target memory corresponding to the memory size requirement in a number of target shared memory areas; return the memory information corresponding to the target memory to any slave node; The slave node is used to, when obtaining the memory information returned by the master node, judge the memory type of the target memory based on the memory information. If the memory requirement process of itself executes a preset memory mapping method, map the target memory to the virtual address space of the memory requirement process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory requirement process can use the target memory.

[0047] In this embodiment, if the master node in the memory sharing system receives a memory application request from any slave node, it can apply for a target memory in the target memory sharing area according to the corresponding memory size requirement of the request, and feedback the memory information of the target memory to the slave node that sends the request, so that the slave node can judge the corresponding memory type according to the memory information. And if the process corresponding to the slave node executes the memory mapping method, it can map the target memory to the virtual address space of the process according to the offset corresponding to the target memory, so that the process can use the allocated target memory. In this way, the hosts in the memory sharing system can be divided into a master node and slave nodes. When the slave nodes apply for memory, they send memory application requests to the master node. The master node performs unified resource management on the entire shared memory and responds to the memory application requests of the slave nodes, and at the same time provides a corresponding synchronization mechanism to ensure data consistency when multiple hosts access the shared memory.

[0048] In some embodiments, the cross-host memory sharing system is further used for: Divide the shared memory area according to preset area division parameters to obtain a number of divided memory areas; Determine the current memory application frequency, and set memory division parameters based on the memory application frequency; Divide a number of divided memory areas according to the memory division parameters and preset memory pages respectively, so as to divide a number of divided memory areas into corresponding a number of memory blocks respectively.

[0049] In some embodiments, the cross-host memory sharing system is further used for: Sort a number of memory blocks corresponding to a number of divided memory areas respectively based on a preset size order, and determine the memory block information corresponding to a number of memory blocks respectively; Store the memory block information in the corresponding a number of divided memory areas to obtain a number of target shared memory areas; the memory block information is the quantity information and memory size information of the memory blocks.

[0050] In some embodiments, the master node is specifically used for: If the master node receives a memory application request from a target slave node, it parses the memory application request to determine the memory size requirement corresponding to the memory application request; Match the memory size requirement with a number of target shared memory regions to determine a target memory region in the number of target shared memory regions that matches the memory size requirement; Determine whether there is free memory in the target memory region that meets the memory size requirement. If there is free memory that meets the memory size requirement, split out the target memory that meets the memory size requirement from the free memory based on a preset memory page; If there is no free memory in the target memory region that meets the memory size requirement, determine the target memory that meets the memory size requirement based on the adjacent region of the target memory region.

[0051] In some embodiments, the master node is specifically configured to: Extract a target memory block that meets the memory size requirement from the target memory region, and perform page alignment splitting on the target memory block through a preset memory page to obtain a number of split memories; Extract the target free memory that meets the memory size requirement from the number of split memories, and use the target free memory as the target memory.

[0052] In some embodiments, the master node is specifically configured to: If there is no free memory in the target memory region that meets the memory size requirement, use the next memory region adjacent to the target memory region as the current target memory region, and jump to the step of determining whether there is free memory in the target memory region that meets the memory size requirement to obtain the target memory; If the target memory is not obtained, use the previous memory region adjacent to the target memory region as the current target memory region, and jump to the step of determining whether there is free memory in the target memory region that meets the memory size requirement to obtain the target memory.

[0053] In some embodiments, the master node is specifically configured to: Determine the offset corresponding to the target memory, and use the offset and the number of shared memory regions corresponding to the target memory as the memory information corresponding to the target memory, and return the memory information to the target slave node so that the target slave node can determine whether the target memory corresponds to a single shared memory region or multiple shared memory regions based on the memory information.

[0054] In some embodiments, the slave node is specifically configured to: If the memory type indicates that the target memory corresponds to a single shared memory region and the memory requirement process executes a preset memory mapping method, determine the physical address of the target memory in the target shared memory region based on the offset corresponding to the target memory; Map the target memory to the virtual address space of the memory demand process corresponding to the target slave node through the physical address, so that the memory demand process can use the target memory.

[0055] In some embodiments, the slave node is specifically used for: If the memory type indicates that the target memory corresponds to multiple shared memory regions, create a target file, and save the memory information as a page fault offset to the private space of the target file; If the memory demand process executes a preset memory mapping method, map the page fault offset in the private space to the virtual address space of the memory demand process, and find the physical address corresponding to the target memory through the page fault offset, so that the memory demand process can use the target memory based on the physical address.

[0056] In some embodiments, the master node is further used for: Record the target start address of the memory region to be released and the target end address after page alignment of the memory region to be released in several target shared memory regions; Detect the free state of the page corresponding to the target start address through a preset bitmap. If it is free, find the end address that is the same as the target start address, and merge the first memory region to be merged corresponding to the end address with the memory region to be released to obtain a first merged region; And / or, detect the free state of the page corresponding to the target end address through a preset bitmap. If it is free, find the start address that is the same as the target end address, and merge the second memory region to be merged corresponding to the start address with the memory region to be released to obtain a second merged region.

[0057] In some embodiments, the master node is further used for: If the target memory is currently occupied and a target memory application request for obtaining the target memory is received from a memory - calling slave node among several slave nodes, set the target memory to an unavailable state through a preset interface, and feedback memory - inaccessible information to the memory - calling slave node; the memory - calling slave node is a slave node different from the target slave node.

[0058] In some embodiments, the master node is further used for: If the target memory has been released, set the target memory to an available state through a preset interface; If the master node receives a target memory application request sent by the memory - calling slave node again, allocate the target memory to the memory - calling slave node.

[0059] For the description of the features in the embodiments corresponding to the cross - host memory sharing system, reference can be made to the relevant description of the embodiments corresponding to the cross - host memory sharing method, which will not be elaborated here one by one.

[0060] An embodiment of the present application further provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any of the above-described embodiments of the cross-host memory sharing method.

[0061] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in any of the above-described embodiments of the cross-host memory sharing method when running.

[0062] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as USB flash drives, read-only memories (ROM), random access memories (RAM), mobile hard disks, magnetic disks, or optical discs that can store computer programs.

[0063] An embodiment of the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the cross-host memory sharing method.

[0064] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the cross-host memory sharing method.

[0065] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0066] The above has introduced in detail a cross-host memory sharing method, system, device, and medium provided by the present application. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A method for cross-host memory sharing, characterized in that The memory sharing system includes a master node and several slave nodes. The memory sharing method is applied to the master node and includes: Receiving a memory application request from a target slave node; the target slave node is any one of the several slave nodes; Determining the memory size requirement corresponding to the memory application request; Matching a target memory corresponding to the memory size requirement in several target shared memory regions; Returning the memory information corresponding to the target memory to the target slave node, so that the target slave node can judge the memory type of the target memory based on the memory information. If the memory requirement process of the target slave node executes a preset memory mapping method, then based on the memory type and the offset corresponding to the target memory in the memory information, map the target memory to the virtual address space of the memory requirement process, so that the memory requirement process can use the target memory.

2. The method for cross-host memory sharing according to claim 1, wherein Before receiving the memory application request from the target slave node, it further includes: Dividing the shared memory region according to preset region division parameters to obtain several divided memory regions; Determining the current memory application frequency and setting memory division parameters based on the memory application frequency; Dividing the several divided memory regions according to the memory division parameters and preset memory pages respectively, so as to divide the several divided memory regions into corresponding several memory blocks respectively.

3. The method for cross-host memory sharing according to claim 2, wherein Before receiving the memory application request from the target slave node, it further includes: Sorting the several memory blocks corresponding to the several divided memory regions respectively based on a preset size order, and respectively determining the memory block information corresponding to the several memory blocks; Storing the memory block information in the corresponding several divided memory regions to obtain several target shared memory regions; the memory block information is the quantity information and memory size information of the memory blocks.

4. The method for cross-host memory sharing according to claim 2, wherein The matching a target memory corresponding to the memory size requirement in several target shared memory regions includes: Matching the memory size requirement with the several target shared memory regions to determine a target memory region in the several target shared memory regions that matches the memory size requirement; Determining whether there is free memory in the target memory region that meets the memory size requirement. If there is free memory that meets the memory size requirement, then splitting out a target memory that meets the memory size requirement from the free memory based on the preset memory page; If there is no free memory in the target memory region that meets the memory size requirement, then determining a target memory that meets the memory size requirement based on the adjacent region of the target memory region.

5. The method for cross-host memory sharing according to claim 4, wherein The splitting out a target memory that meets the memory size requirement from the free memory based on the preset memory page includes: Extracting a target memory block that meets the memory size requirement from the target memory region, and performing page alignment splitting on the target memory block through the preset memory page to obtain several split memories; Extracting a target free memory that meets the memory size requirement from the several split memories, and using the target free memory as the target memory.

6. The method for cross-host memory sharing according to claim 4, wherein If there is no free memory in the target memory area that meets the memory size requirement, determining the target memory that meets the memory size requirement based on the adjacent area of the target memory area includes: If there is no free memory in the target memory area that meets the memory size requirement, taking the next memory area adjacent to the target memory area as the current target memory area, and jumping to the step of determining whether there is free memory in the target memory area that meets the memory size requirement to obtain the target memory; If the target memory is not obtained, taking the previous memory area adjacent to the target memory area as the current target memory area, and jumping to the step of determining whether there is free memory in the target memory area that meets the memory size requirement to obtain the target memory.

7. The method for cross-host memory sharing according to claim 1, wherein Returning the memory information corresponding to the target memory to the target slave node, so that the target slave node can determine the memory type of the target memory based on the memory information, includes: Determining the offset corresponding to the target memory, and taking the offset and the number of shared memory areas corresponding to the target memory as the memory information corresponding to the target memory, and returning the memory information to the target slave node, so that the target slave node can determine whether the target memory corresponds to a single shared memory area or multiple shared memory areas based on the memory information.

8. The cross-host memory sharing method according to claim 1, wherein If the memory requirement process of the target slave node executes a preset memory mapping method, mapping the target memory to the virtual address space of the memory requirement process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory requirement process can use the target memory, includes: If the memory type indicates that the target memory corresponds to a single shared memory area, and the memory requirement process executes a preset memory mapping method, determining the physical address of the target memory in the target shared memory area based on the offset corresponding to the target memory; Mapping the target memory to the virtual address space of the memory requirement process corresponding to the target slave node through the physical address, so that the memory requirement process can use the target memory.

9. The method for cross-host memory sharing according to claim 1, wherein, If the memory requirement process of the target slave node executes a preset memory mapping method, mapping the target memory to the virtual address space of the memory requirement process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory requirement process can use the target memory, includes: If the memory type indicates that the target memory corresponds to multiple shared memory areas, creating a target file, and saving the memory information as a page fault offset to the private space of the target file; If the memory requirement process executes the preset memory mapping method, mapping the page fault offset in the private space to the virtual address space of the memory requirement process, and finding the physical address corresponding to the target memory through the page fault offset, so that the memory requirement process can use the target memory based on the physical address.

10. The method for cross-host memory sharing according to claim 1, wherein Further includes: Record the target start address of the memory area to be released in the several target shared memory areas and the target end address after page alignment of the memory area to be released; Detect the free status of the page corresponding to the target start address through a preset bitmap. If it is free, find the end address that is the same as the target start address, and merge the first memory area to be merged corresponding to the end address with the memory area to be released to obtain a first merged area; And / or, detect the free status of the page corresponding to the target end address through a preset bitmap. If it is free, find the start address that is the same as the target end address, and merge the second memory area to be merged corresponding to the start address with the memory area to be released to obtain a second merged area.

11. The method for cross-host memory sharing according to any one of claims 1 to 10, characterized in that Further includes: If the target memory is currently occupied and a target memory application request for obtaining the target memory is received from a memory call slave node among the several slave nodes, set the target memory to an unavailable state through a preset interface, and feedback memory inaccessible information to the memory call slave node; The memory call slave node is a slave node different from the target slave node.

12. The method for cross-host memory sharing according to claim 11, wherein Further includes: If the target memory has been released, set the target memory to an available state through the preset interface; If the target memory application request sent by the memory call slave node is received again, allocate the target memory to the memory call slave node.

13. A cross-host memory sharing system, characterized in that, Includes a master node and several slave nodes; The master node is used to receive a memory application request from any slave node, and determine the memory size requirement corresponding to the memory application request; match a target memory corresponding to the memory size requirement in several target shared memory areas; return the memory information corresponding to the target memory to the any slave node; The slave node is used to, when obtaining the memory information returned by the master node, judge the memory type of the target memory based on the memory information. If the memory requirement process of itself executes a preset memory mapping method, map the target memory to the virtual address space of the memory requirement process based on the memory type and the offset corresponding to the target memory in the memory information, so that the memory requirement process can use the target memory.

14. An electronic device, characterized in that, Includes: A memory for storing a computer program; A processor for implementing the steps of the cross-host memory sharing method according to any one of claims 1 to 12 when executing the computer program.

15. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein the computer program implements the steps of the cross-host memory sharing method according to any one of claims 1 to 12 when executed by a processor.

Images