A server memory expansion method and electronic device

Abstract

This application discloses a server memory expansion method and electronic device, relating to the field of server technology. It initializes a memory resource management mechanism and sends probe signals to each server node. Based on the memory resource information fed back by each server node, a memory resource table is generated; based on the transmission time of data packets between nodes, a node inter-transmission speed table is constructed. By maintaining the memory resource table and the node inter-transmission speed table, the master node can determine the lending node when a task node sends a memory expansion request and send the lending node's node information to the task node, enabling the task node to access the lending node's memory. The master node updates the memory resource table based on the node memory status information fed back by the lending node, thereby ensuring the accuracy of the memory resource table. By maintaining the memory resource table and the node inter-transmission speed table, efficient integration and dynamic management of memory resources can be achieved, improving memory expansion capabilities and data access speed.

Description

Technical Field

[0001] This application relates to the field of server technology, and in particular to a server memory expansion method and electronic device. Background Technology

[0002] In today's era of rapid technological advancement, the rapid progress in large-scale modeling, high-performance computing, and game development is reshaping computing demands. Applications in these fields pose unprecedented challenges to computing power and storage requirements.

[0003] Training and inference of large models require massive memory resources, while high-performance computing demands extremely fast access speeds when processing large amounts of data. Meanwhile, game development, with its real-time rendering and complex scenes, also places stringent demands on memory and bandwidth. To support these advanced applications, the memory scalability of traditional server architectures often proves insufficient, failing to provide adequate speed when faced with large-scale data transfers.

[0004] It is evident that improving memory scalability and providing faster data access speeds are problems that need to be solved by those skilled in the art. Summary of the Invention

[0005] This application provides a server memory expansion method and electronic device to at least solve the problem of insufficient memory expansion capability and inability to provide sufficient speed in related technologies.

[0006] This application provides a server memory expansion method, applicable to master nodes, the method including:

[0007] Initialize the memory resource management mechanism and send probe signals to each server node;

[0008] Based on the memory resource information returned by each server node, a memory resource table is generated;

[0009] Based on the transmission time of data packets between nodes, construct a table of node transmission speeds;

[0010] Upon receiving a memory expansion request from a task node, the system determines the lending node that will lend memory to the task node based on the memory resource table, the node inter-transfer speed table, and the memory capacity carried in the memory expansion request, and sends the node information of the lending node to the task node.

[0011] Update the memory resource table based on the node memory status information reported by the lending node.

[0012] This application also provides a server memory expansion device suitable for master nodes. The device includes an initialization unit, a generation unit, a construction unit, a determination unit, a sending unit, and an update unit.

[0013] The initialization unit is used to initialize the memory resource management mechanism and send probe signals to each server node.

[0014] The generation unit is used to generate a memory resource table based on the memory resource information fed back by each server node;

[0015] The building unit is used to build a table of inter-node transmission speed based on the transmission time of data packets between nodes;

[0016] The determining unit is used to determine the lending node that will lend memory to the task node based on the memory resource table, the node inter-transfer speed table, and the memory capacity carried in the memory expansion request when a memory expansion request is received from the task node.

[0017] The sending unit is used to send the node information of the lending node to the task node;

[0018] The update unit is used to update the memory resource table based on the node memory status information fed back by the lending node.

[0019] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described server memory expansion methods when executing the computer program.

[0020] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described server memory expansion methods.

[0021] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described server memory expansion methods.

[0022] This application initializes a memory resource management mechanism and sends probe signals to each server node. The memory resources of each server node and the speed of data transfer between server nodes are crucial factors affecting memory allocation. Therefore, this application generates a memory resource table based on the memory resource information fed back by each server node and constructs a node transfer speed table based on the data packet transmission time between nodes. Upon receiving a memory expansion request from a task node, the application determines the lending node to lend memory to the task node based on the memory resource table, the node transfer speed table, and the memory capacity carried in the memory expansion request. The lending node's node information is then sent to the task node so that the task node can access the lending node's memory based on the node information. After a lending node successfully lends memory to a task node, its node memory status information changes. The lending node then feeds back its memory status information to the master node. The master node updates the memory resource table based on the node memory status information fed back by the lending node, thereby ensuring the accuracy of the memory resource table. In this technical solution, each server node, through the initialization of the memory resource management mechanism, ensures normal communication with other server nodes, enabling adaptive and efficient multi-node interconnection. By maintaining a memory resource table and a node transfer speed table, the master node can achieve efficient integration and dynamic management of memory resources among server nodes, solving the current challenges of memory expansion and high-speed access. The memory expansion method provided in this application significantly enhances the computing power of server nodes, providing strong support for fields such as large-scale models, high-performance computing, and game development. Attached Figure Description

[0023] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A network architecture diagram for server memory expansion provided in this application embodiment;

[0025] Figure 2 A flowchart illustrating a server memory expansion method provided in this application embodiment;

[0026] Figure 3 A schematic diagram of a training set provided for an embodiment of this application;

[0027] Figure 4 A flowchart illustrating a method for lending memory, provided in an embodiment of this application;

[0028] Figure 5 A network architecture diagram of a consistent interconnection extension unit provided in this application embodiment;

[0029] Figure 6 A schematic diagram illustrating an update of an extensible memory mapping table provided in an embodiment of this application;

[0030] Figure 7 This is a schematic diagram of the structure of a server memory expansion device provided in an embodiment of this application. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0032] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0033] In today's rapid development of artificial intelligence, the rise of large-scale models has attracted widespread attention and research. Large-scale models are highly favored for their powerful language understanding and generation capabilities; however, their training and inference processes place extremely demanding requirements on computing resources, especially in terms of memory expansion and high-speed access. This challenge not only affects the performance of the models but also poses new requirements for the design and implementation of the entire technological ecosystem.

[0034] First, large models typically contain billions or even tens of billions of parameters, meaning they require a significant amount of memory during runtime. As model size continues to increase, traditional hardware architectures struggle to meet their memory demands. Achieving efficient model training and inference under limited hardware conditions remains a challenge. To address this, new storage architectures are being explored, such as distributed computing and memory sharing technologies, to achieve higher memory utilization and faster data access speeds. Second, high-speed memory access is a key factor for the efficient operation of large models. The training and inference processes of large models often require frequent data reads and writes, placing stringent demands on memory bandwidth and latency. However, existing storage technologies often fail to provide sufficient speed when faced with large-scale data transfers. This has led researchers to explore next-generation storage media, such as non-volatile memory (NVM) and three-dimensional NAND flash memory (3D NAND), aiming to achieve faster access speeds while maintaining data persistence.

[0035] Furthermore, with the continuous expansion of large-scale model applications, such as natural language processing, computer vision, and speech recognition, how to achieve efficient inference on different devices has become an urgent problem to be solved. In the context of the increasing prevalence of edge computing and mobile devices, how to compress and optimize massive models to enable them to run efficiently in resource-constrained environments has become a focus of attention.

[0036] The emergence of Compute Express Link (CXL) technology has opened up new possibilities for server memory expansion. CXL is a high-bandwidth, low-latency interconnect standard that effectively connects Central Processing Units (CPUs) with devices such as accelerators and memory. Combined with memory pooling technology, CXL can not only integrate the memory resources of multiple servers into a shared pool but also dynamically allocate memory, improving resource utilization. The implementation of this technology enables servers to flexibly allocate memory when facing large-scale models and high-performance computing tasks, significantly improving performance.

[0037] Therefore, this application proposes a server memory expansion method, apparatus, electronic device, and storage medium. Each server node deploys a CXL expansion unit. The server node, through an initialization memory resource management mechanism, ensures normal communication with other server nodes, enabling adaptive and efficient multi-node interconnection. The master node, by maintaining a memory resource table and a node inter-node transfer speed table, can achieve efficient integration and dynamic management of memory resources among server nodes, solving the current challenges of memory expansion and high-speed access, improving the computing power of server nodes, and providing strong support for large-scale models, high-performance computing, and game development. Through memory pooling, the allocation of computing resources becomes more flexible, allowing users to dynamically adjust memory according to actual needs, thereby solving the problems of memory expansion and access speed, and meeting the complex requirements of modern applications.

[0038] Figure 1 This application provides a network architecture diagram for server memory expansion. Each server node includes a hard disk, memory, a central processing unit, and a consistency interconnect extension unit. The consistency interconnect extension unit is a software module deployed within the server node. The consistency interconnect extension unit of each server node enables interconnection between server nodes through a high-speed consistency network (Remote Direct Memory Access, RDMA).

[0039] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] Figure 2 A flowchart of a server memory expansion method provided in this application embodiment, applicable to a master node, includes:

[0041] S201: Initialize the memory resource management mechanism and send probe signals to each server node.

[0042] The server memory expansion scheme provided in this application is applicable to systems containing multiple server nodes, which can be simply referred to as nodes. To implement memory management, one server node can be designated as the internal network master server node, or simply the master node. The purpose of the master node is to build the memory management system for the entire internal network; it is not required that all programs run on the master node. The master node is configurable and transferable.

[0043] During server startup or maintenance, each server node initializes its memory resource management mechanism and configures its network parameters through the built-in Consistent Interconnect Interface (CXL interface).

[0044] Taking the master node as an example, the master node configures network parameters through the built-in consistent interconnect interface to facilitate communication with other server nodes through the network parameters; the network parameters may include network topology, node address and communication protocol.

[0045] S202: Generate a memory resource table based on the memory resource information returned by each server node.

[0046] To help the master node understand the memory resource status of each server node, the master node can maintain a memory resource table.

[0047] In practice, the master node can send probe signals to other server nodes in the network via the CXL interface. These probe signals contain the master node's identification information, enabling other server nodes to recognize and respond. Upon receiving the probe signals, other server nodes can, according to a pre-defined response strategy, relay their own memory resource information to the master node.

[0048] The preset response strategy includes the necessary information for memory resource information, which may include a combination of node identifier, memory type, memory capacity, memory bandwidth, remaining memory capacity, node memory status, and borrowed memory status.

[0049] The master node can receive memory resource information from each server node; construct a memory resource table according to the set data format; and record the memory resource information of each server node into the memory resource table.

[0050] The data format may include a combination of node identifier, basic memory information, remaining memory capacity, node memory status, and borrowed memory status. Basic memory information may include memory type, memory capacity, and memory bandwidth.

[0051] The node identifier can be a node address, which can be an Internet Protocol (IP) address. The memory type can be Double Data Rate Fourth Generation SDRAM (DDR4) or Double Data Rate Fifth Generation SDRAM (DDR5), etc.

[0052] To quickly and easily distinguish different server nodes, a node number can be set for each server node in the memory resource table. For example, if there are 10 server nodes in total, the node numbers can be 1 to 10.

[0053] The memory resource table constructed by the master node can be stored in the CXL extension unit. Table 1 is a memory resource table provided in an embodiment of this application.

[0054] Table 1 Memory Resource Table

[0055]

[0056] Table 1 only shows the data types contained in each row. In actual applications, the system contains multiple server nodes, and the memory resource table records the resource information of each of these server nodes.

[0057] The node memory status in the memory resource table is synchronized with the CXL memory status of each server node; the borrowed memory status of each server node records the amount of memory that the node borrows from other nodes.

[0058] Each server node initializes its memory state and can store the memory state information of all free memory segments of the current node in the form of a memory state table for local or remote CXL memory access.

[0059] In the implementation, each server node maintains a node memory state table through the memory manager of the CXL extension unit. This table records the node's memory state information. The memory state information may include the starting address of a free memory segment, its segment size, and a segment lendable flag. The segment lendable flag indicates whether the free memory segment can be lent to other nodes. The memory state table for each server node is shown below:

[0060] Table 2 Node Memory Status Table

[0061]

[0062] The node memory status table contains the starting addresses of all free memory segments of that node, along with their segment sizes and borrowable flags. Table 2 uses n free memory segments as an example; the value of n determines how many free memory segments a server node contains. The numbers in Table 2 are only used to distinguish different memory segments and do not impose any other restrictions.

[0063] There are two possible values ​​for the segment lending flag: when the segment lending flag is 1, it means that the free memory segment can be lent to other nodes; when the segment lending flag is 0, it means that the free memory segment cannot be lent to other nodes.

[0064] Since memory fragmentation affects program running efficiency, in order to avoid excessively fragmented available memory, in practical applications, the available memory flag can be set to 1 when the fragment size is greater than 1G, and set to 0 when the fragment size is less than or equal to 1G.

[0065] In Table 2, the total free memory capacity is the sum of the capacities of all free memory segments, and the total available free memory capacity is the sum of the capacities of all free memory segments whose available flag is 1.

[0066] Because the memory of a node goes through a process of being occupied and released, the node memory state table maintained by each node will change. When the memory state information in the node memory state table changes, the node can transmit the memory state information to the master node so that the master node can update the memory resource table in a timely manner.

[0067] The memory lending status record shows the amount of memory each server node has lent to other nodes. To facilitate memory management, each server node can maintain a memory lending status table, which includes the IP address and capacity of the lent memory. Table 3 below shows the memory lending status table for a single server node.

[0068] Table 3. Loaned Memory Status Table

[0069] .

[0070] S203: Construct a node-to-node transmission speed table based on the data packet transmission time between nodes.

[0071] Considering that the time between different nodes may not be completely synchronized in practical applications, in order to ensure the accuracy of the data packet transmission time between nodes, the master node can determine the transmission time of the test packet by sequentially transmitting the timestamp, test packet and timestamp to other nodes.

[0072] Taking any one of the server nodes, i.e., the target server node, as an example, the master node can send a first timestamp to the target server node. After sending the first timestamp, it sends a test packet and its message digest to the target server node. After sending the test packet, it sends a second timestamp to the target server node. It then receives the target transmission time from the target server node; the target transmission time is the difference between the second timestamp and the first timestamp received by the target server node. Based on the data length of the test packet and the target transmission time, the mutual transmission speed between the master node and the target server node is determined.

[0073] The method of determining the mutual transmission speed is based on the sequential transmission of timestamps, test packets, and timestamps from the master node to the target server node. Each server node can also determine the mutual transmission speed between nodes by sequentially transmitting timestamps, test packets, and timestamps to other server nodes.

[0074] When the master node receives a test packet from another server node, it performs a hash operation on the test packet to obtain a hash value. It then checks if the hash value matches the message digest carried in the test packet. If the hash value matches the message digest, the data transmission is complete and consistent. At this point, the master node can report the transmission time to other server nodes; the transmission time is the difference between two timestamps transmitted by other server nodes.

[0075] In the implementation, the master node can send a verification data packet and its message digest to each of the other nodes. The verification data packet can be user-defined and is typically about 1GB in size. The verification data packet is encoded using a Message-Digest Algorithm 5 (MD5) to obtain the message digest, i.e., the MD5 encoding. The receiving node verifies the correctness of this message digest. Afterward, the process is repeated for each node, sending the data packet sequentially to the remaining nodes, thus completing the transmission of the speed table between nodes.

[0076] Taking any two server nodes, such as node A and node B, as an example, considering that the local times of node A and node B may not be consistent, the same node sends the timestamps. Taking the test of the data packet transmission speed from node A to node B as an example, node A first sends a timestamp t1 to node B, and then sends the test packet. After sending the test packet, it sends a timestamp t2. Then, node B locally verifies whether the MD5 hash of the test packet is correct. If incorrect, this step is repeated; if correct, node B calculates the transmission time, i.e., t2-t1, and feeds the transmission time back to the master node. The quotient of the test packet's data length and the transmission time is the transmission speed, and the master node fills the transmission speed into the corresponding position in the node inter-transmission speed table.

[0077] To facilitate the management of data packet transmission time between nodes, the master node can construct a node-to-node transmission speed table, as shown in Table 4. The node-to-node transmission speed table is stored in memory.

[0078] Table 4 Node-to-Node Transmission Speed ​​Table

[0079]

[0080] In Table 4, Node 1, Node 2, and Node 3 are the node serial numbers, corresponding to the node serial numbers in the memory resource table. The i-th row and j-th column of Table 4 represents the transmission speed from server node i to server node j, for example, V in Table 4... 2,1 This indicates the transmission speed from node 2 to node 1. The areas marked with diagonal lines in Table 4 indicate that the transmission speed does not need to be calculated.

[0081] S204: Upon receiving a memory expansion request from a task node, determine the lending node that will lend memory to the task node based on the memory resource table, the node inter-transfer speed table, and the memory capacity carried in the memory expansion request, and send the node information of the lending node to the task node.

[0082] In this embodiment, the node characteristic data of each server node can be determined based on the memory resource table and the node inter-transfer speed table. The node characteristic data of each server node is input into a graph neural network model to obtain the probability value of each server node except for the task node. The node lending priority is determined according to the memory capacity carried in the memory expansion request, the probability value of each server node, and the remaining memory capacity. The first server node in the node lending priority ranking is selected as the lending node.

[0083] Node characteristic data can include upload speed, download speed, remaining memory capacity, borrowed memory capacity, and total memory borrowed by task nodes.

[0084] After obtaining the probability value of each server node, the server nodes can be sorted in descending order according to the probability value, and server nodes with remaining memory capacity less than the memory capacity carried in the memory expansion request can be deleted to obtain the node lending priority sort.

[0085] The higher the probability value of a server node, the more suitable its memory is for lending to a task node. Therefore, after sorting the nodes in descending order of probability value, the first server node in the node lending priority sort can be used as the lending node.

[0086] S205: Update the memory resource table based on the node memory status information fed back by the lending node.

[0087] After receiving the node information from the lending node, the task node can send a memory borrowing request to the lending node. The lending node verifies whether the memory status of its local CXL expansion unit meets the requirements. For example, if the total available free memory capacity is greater than the memory capacity carried by the memory borrowing request, it means that the memory status meets the requirements.

[0088] When the memory status of the lending node meets the requirements, the lending node will determine the memory segment that can be lent to the task node based on the memory capacity required by the task node, and feed back the starting address and length of the memory segment to the task node so that the task node can successfully access the memory of the lending node.

[0089] Since the lending node lends its memory to the task node, the lending node's memory status information will change. At this time, the lending node will report the memory status information to the master node. Based on this memory status information, the master node updates the memory resource table, ensuring the timeliness of information updates in the memory resource table and guaranteeing its accuracy.

[0090] As can be seen from the above technical solution, an initial memory resource management mechanism is established, and probe signals are sent to each server node. The memory resources of each server node and the speed of inter-server data transfer are important factors affecting memory allocation. Therefore, in this application, a memory resource table is generated based on the memory resource information fed back by each server node; and a node inter-server data transfer speed table is constructed based on the transmission time of data packets between nodes. Upon receiving a memory expansion request from a task node, the lending node is determined based on the memory resource table, the node inter-server data transfer speed table, and the memory capacity carried in the memory expansion request. The node information of the lending node is then sent to the task node so that the task node can access the memory of the lending node based on the node information. When the lending node successfully lends memory to the task node, its node memory status information changes. The lending node feeds back its memory status information to the master node, and the master node updates the memory resource table based on the node memory status information fed back by the lending node, thereby ensuring the accuracy of the memory resource table. In this technical solution, each server node, through the initialization of the memory resource management mechanism, ensures normal communication with other server nodes, enabling adaptive and efficient multi-node interconnection. By maintaining a memory resource table and a node transfer speed table, the master node can achieve efficient integration and dynamic management of memory resources among server nodes, solving the current challenges of memory expansion and high-speed access. The memory expansion method provided in this application significantly enhances the computing power of server nodes, providing strong support for fields such as large-scale models, high-performance computing, and game development.

[0091] In practical applications, after the first server node in the node lending priority sort is selected as the lending node, there may be a situation where the memory state of the lending node does not meet the lending requirements. In this case, the lending node will send a rejection signal to the master node.

[0092] When the master node receives a rejection signal from the lending node, it designates the next server node in the node lending priority ranking as the new lending node. The next server node is the server node immediately following the adjacent lending node.

[0093] When a task node needs to borrow a large amount of memory, there may be a situation where the memory capacity exceeds the remaining memory capacity of a single lending node. In this case, the memory of a single lending node can be allocated to the task node first, and a prompt message can be sent to the task node so that the task node can send a memory expansion request again if the memory is insufficient.

[0094] In the specific implementation, after taking the first server node in the node lending priority sort as the lending node, it can be determined whether the remaining memory capacity of the first server node in the node lending priority sort is less than the memory capacity carried in the memory expansion request.

[0095] If the remaining memory capacity of the first server node in the node lending priority sort is less than the memory capacity carried in the memory expansion request, it means that the allocated borrowed memory cannot meet the memory requirements of the task node. At this time, the master node can send a prompt message to the task node so that the task node can send a memory expansion request again if memory is insufficient.

[0096] In this embodiment, memory is flexibly allocated to task nodes based on the lending status of lending nodes. If the memory status of a lending node does not meet the lending requirements, the next server node in the node lending priority ranking can be used as the new lending node. If the remaining memory capacity of a lending node is less than the memory capacity required by the task node, multiple allocations of lending nodes can be made to the task node to meet its memory requirements for task execution.

[0097] In this embodiment of the application, a graph neural network model can be used to analyze the node feature data of each server node to determine the probability value of each server node.

[0098] Before using a graph neural network (GNN) model, it needs to be trained. In the specific implementation, the GNN model memory allocator of the CXL extension unit of the master node records all historical memory allocation data to build the training set of the GNN model.

[0099] Figure 3 This diagram illustrates a training set provided in an embodiment of this application. The training set may include two parts: input data and label data. The input data and node feature data are constructed in the same way. The input data contains multiple samples. Figure 3This example uses K samples. Each sample contains training input data and training label data for each of the n nodes. The training input data includes upload speed, download speed, remaining memory capacity, total borrowed memory, and total borrowed memory for the task nodes. The training label data is used to characterize whether the allocation was ultimately successful: 1 for successful allocation, -1 for failure, and 0 for no allocation to the node.

[0100] The graph neural network model is trained using the training input data and training label data contained in the training set; the parameters of the graph neural network model are optimized according to the gradient descent algorithm, and the weights of the graph neural network model are output to obtain the trained graph neural network model.

[0101] To improve the fit of the graph neural network model with the current application scenario, a memory allocation count D can be set. Every D additional memory allocations will retrain the graph neural network model and update its weights.

[0102] Each time the master node determines a lending node, it is equivalent to completing one memory allocation.

[0103] In the specific implementation, after sending the node information of the lending node to the task node, the memory allocation count is incremented by one for each memory allocation completed; initially, the memory allocation count is zero. When the memory allocation count reaches the set threshold, the sample data is updated using the node feature data and label data corresponding to the memory allocation count; the graph neural network model is then retrained using the updated sample data.

[0104] The threshold number of times is D, and the value of D can be flexibly set based on actual needs. There is no limit to this value.

[0105] The sample data can be updated by using the node feature data and label data corresponding to the number of memory allocations. This can be done by deleting the first D data entries from the original sample data and adding the node feature data and label data corresponding to the D memory allocations to the original sample data, thus obtaining the updated sample data.

[0106] In this embodiment, by updating the sample data based on historical memory allocation, and using the updated sample data to train the graph neural network model, the graph neural network model is made more compatible with the current application scenario. This allows the output node borrowing priority sorting to better meet the scenario requirements of the current application scenario and improves the accuracy of memory resource allocation.

[0107] Figure 4 A flowchart of a method for lending memory provided in this application embodiment, the method including:

[0108] S401: If it is a lending node, construct a memory address table.

[0109] In practical applications, any server node can be both a task node and a lending node.

[0110] Regardless of which server node acts as the lending node, the process for lending memory is similar. Taking the master node as an example, when the master node determines that it is a lending node based on the memory resource table, the node inter-node transfer speed table, and the memory capacity carried in the memory expansion request, it can first construct a memory address table to facilitate feedback to the task nodes on the memory segments it can lend.

[0111] The memory address table may include the starting address of a segment, the segment length, and the total length; initially, the memory address table has no data and the total length is zero. Table 5 below shows a memory address table provided in an embodiment of this application.

[0112] Table 5 Memory Address Table

[0113] .

[0114] S402: Upon receiving a memory borrowing request from a task node, traverse all available memory in the memory status information and obtain the current available memory length.

[0115] The memory status information includes the starting address of each free segment, its segment length, and the available segment flag.

[0116] Nodes can record memory status information through a node memory status table. By referring to the node memory status table shown in Table 2, all free memory segments with the borrowable flag set to 1 can be identified for that node.

[0117] Generally, the data in the node memory status table can be sorted in descending order of free memory segment capacity. In practical applications, the memory status table can be traversed. Initially, the first free memory segment is retrieved, and its capacity is the current available memory length. If the capacity of the first free memory segment is insufficient for the task node's memory requirements, its capacity is used as the total length, and the capacity of the next free memory segment is read and used as the current available memory length. This process continues until the total length reaches the required memory length for the task node, at which point the operation ends.

[0118] S403: If the sum of the current available memory length and the total length is less than the memory capacity carried in the memory expansion request, record the current available memory length and its corresponding starting address to the memory address table, and update the total length and memory status information.

[0119] Initially, the memory address table contains no data and has a total length of zero. Each time a segment's starting address and length are added, the total length is updated. The total length is the sum of the lengths of all segments recorded in the memory address table.

[0120] If the sum of the current available memory length and the total length is less than the memory capacity carried in the memory expansion request, the current available memory length and its corresponding starting address can be recorded in the memory address table, and the total length and memory status information can be updated.

[0121] Since the sum of the current available memory length and the total length is less than the memory capacity carried in the memory expansion request, it means that the currently lent memory cannot meet the memory requirements of the task node. At this time, it is necessary to read the next data from the memory status table. The capacity of the free memory segment corresponding to the next data is the current available memory length. This process continues until the sum of the current available memory length and the total length is greater than or equal to the memory capacity carried in the memory expansion request, at which point S404 is executed.

[0122] S404: If the sum of the current available memory length and the total length is greater than or equal to the memory capacity carried in the memory expansion request, record the starting address corresponding to the current available memory length in the memory address table, record the difference between the memory capacity carried in the memory expansion request and the total length as the segment length in the memory address table, and update the total length and memory status information.

[0123] If the sum of the current available memory length and the total length is greater than the memory capacity carried in the memory expansion request, it means that it is not necessary to lend out all the capacity of the currently free memory segment; only the difference between the memory capacity carried in the memory expansion request and the total length is needed.

[0124] In the specific implementation, the starting address corresponding to the current available memory length can be recorded in the memory address table, and the difference between the memory capacity carried in the memory expansion request and the total length can be recorded as the segment length in the memory address table; and the total length and memory status information can be updated.

[0125] S405: Send the memory address table to the task node if the total length equals the memory capacity carried in the memory expansion request.

[0126] If the total length equals the memory capacity carried in the memory expansion request, it means that the lending node has determined all the lending memory, and the starting address and segment length of all the lending memory segments have been recorded in the memory address table. At this time, the lending node can send the memory address table to the task node so that the task node can call the lending node's memory based on the segment starting address and segment length recorded in the memory address table.

[0127] In this embodiment, each server node relies on a built-in CXL extension unit to manage memory status and facilitate inter-node interaction. Based on the functions required by the CXL extension unit, it can be divided into a memory manager, a virtual memory pool, virtual memory control, and address translation.

[0128] Figure 5 This application provides a network architecture diagram for a Consistent Interconnect Extension Unit (CIU). The CIU is installed on each server node in the network and includes a memory manager, a virtual memory controller, a virtual memory pool, and an address translator. When the server node is the master node, the CIU on the master node also maintains a memory resource table, a node-to-node transfer speed table, and a graph neural network model. The graph neural network model functions as a memory allocator. Figure 5 The modules and tables within the dashed box are only those available to the main node.

[0129] The Consistent Interconnect Extended Unit (CXL Unit) is used by nodes to request memory during task execution. When a node initiates a memory request, its operating system interacts with the CXL Unit's memory manager. By comparing the memory status with local memory, the CXL Unit prioritizes retrieving memory from local memory and updates the memory status. Subsequently, the virtual memory controller transmits the "total available free memory" from the memory status to the master node's CXL Unit, updating the corresponding node's information in the memory resource table.

[0130] When a task node's program makes a memory request that cannot be satisfied by local memory after comparing it with its own memory status, it requests memory from the cluster and makes a virtualization call. The operation steps are as follows:

[0131] Step 1: The virtual memory controller of the task node CXL unit sends a memory expansion request to the master node CXL unit (hereinafter referred to as the master CXL unit) through the CXL.cache protocol, which includes the memory capacity to be expanded.

[0132] Step 2: The main CXL unit retrieves relevant information from the memory resource table and the node transfer speed table, and inputs it along with the memory capacity required by the task node into the graph neural network model for node resource allocation.

[0133] Step 3: The master CXL unit sends the IP address of the lending node to the task node. The task node then sends a memory borrowing request to the lending node. The lending node verifies whether its own CXL unit's memory status meets the requirement. For example, if the total available free memory capacity of the lending node is greater than or equal to the memory capacity of the task node, it means the task node's memory requirement is met. If the requirement is not met, the lending node sends a rejection signal to the master node. The master node then sorts the nodes according to their lending priority, selects the next node as the lending node, and sends the IP address of the next lending node back to the task node, until a lending node that meets the task node's memory requirement is selected.

[0134] For step 2, when using a graph neural network model for memory allocation, the following steps can be included:

[0135] 2.1: Construct node feature data for each node. The format of the feature data is shown in Table 6.

[0136] Table 6 Node Feature Data Table

[0137] .

[0138] Among them, the upload speed is the speed at which the task node transmits verification data packets to the lending node, extracted from the node inter-transmission speed table; the download speed is the speed at which the lending node transmits verification data packets to the task node, extracted from the node inter-transmission speed table; the remaining memory capacity is the remaining memory capacity of the lending node, extracted from the memory resource table; the total amount of memory lent is the memory capacity that the lending node has lent to other nodes, extracted from the "Lent Memory Status" field of the memory resource table; and the total amount of memory lent by the task node is the memory capacity that the lending node has lent to the current task node, extracted from the "Lent Memory Status" field of the memory resource table.

[0139] 2.2: Input the node feature data of each node (including the node feature data of the task node) into the graph neural network model for prediction.

[0140] In this model, the features of each feature node are initialized with the node feature data from step 2.1, and the edges between feature nodes are initialized with the reciprocal of the sum of the upload and download speeds of the two server nodes.

[0141] 2.3: The graph neural network model inference yields the probability value of each node except the task node. These probability values ​​are then sorted in descending order, and nodes with remaining memory capacity less than the memory capacity required by the current task node are removed, resulting in a node borrowing priority sort.

[0142] In practical applications, if the maximum remaining memory capacity of all nodes is less than the memory capacity required by the task node, then the current memory requirement is updated to min{the maximum remaining memory capacity of all nodes, current memory requirement}. If the lent memory is insufficient, the task node requests memory again.

[0143] 2.4: The first node in the node lending priority sort is the lending node.

[0144] In practical applications, the master node may also be a task node. Taking the case where the master node is a task node as an example, the master node can update the expandable memory mapping table based on the memory address table sent by the lending node. The expandable memory mapping table includes the lending node identifier, the starting address of the segment, and the segment length. When the task to be processed calls virtual memory, it is sent to the target lending node according to the expandable memory mapping table so that the target lending node can decode and process the task.

[0145] In the actual implementation, after receiving the memory address table, the task node can update it in the expandable memory mapping table in the address translator. Figure 6 This is a schematic diagram of an updated scalable memory mapping table provided in an embodiment of this application. The CXL unit translates external addresses into local access addresses through an address memory table. The local CPU treats the virtual memory pool as a local memory call, the difference being that the call addresses are different (the addresses are arranged sequentially after the local memory access addresses according to the request order). This scalable memory mapping table exists in the address translator of the CXL unit and is called when accessing virtual memory. Figure 6 Taking the local memory address range of the task node as 0000-000B and the extended memory address range of the task node as 000B-003B as an example, each record in the extended memory mapping table includes the lending node IP, the fragment start address, and the fragment length.

[0146] When the current task uses virtual memory, the memory manager of the CXL unit sends the data packets that need to be processed to the lending node memory through the address translator. The lending node CXL unit then decodes the data packets after receiving them.

[0147] The data packet format is shown in Table 7 below. The "starting address" field in the data packet represents the actual memory address of the lending node.

[0148] Table 7 Data Packet Format Table

[0149] .

[0150] In this embodiment, the task node can directly access the lent memory by updating the memory address returned by the lending node to its own scalable memory mapping table, thereby improving memory access efficiency.

[0151] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0152] Figure 7 This is a schematic diagram of the structure of a server memory expansion device provided in an embodiment of this application. It is applicable to a master node and includes an initialization unit 71, a generation unit 72, a construction unit 73, a determination unit 74, a sending unit 75, and an update unit 76.

[0153] Initialization unit 71 is used to initialize the memory resource management mechanism and send probe signals to each server node;

[0154] The generation unit 72 is used to generate a memory resource table based on the memory resource information fed back by each server node;

[0155] Construction unit 73 is used to construct a node inter-transmission speed table based on the transmission time of data packets between nodes;

[0156] The determining unit 74 is used to determine the lending node that will lend memory to the task node based on the memory resource table, the node inter-transfer speed table, and the memory capacity carried in the memory expansion request when a memory expansion request is received from the task node.

[0157] The sending unit 75 is used to send the node information of the lending node to the task node;

[0158] Update unit 76 is used to update the memory resource table based on the node memory status information fed back by the lending node.

[0159] In some embodiments, the initialization unit is configured to configure network parameters through a built-in consensus interconnection interface to facilitate communication with other server nodes through the network parameters; wherein, the network parameters include network topology, node address and communication protocol.

[0160] In some embodiments, the generating unit includes a receiving subunit, a building subunit, and a recording subunit;

[0161] The receiving subunit is used to receive memory resource information fed back by each server node;

[0162] The construction sub-unit is used to build a memory resource table according to a set data format; wherein the data format includes a combination of node identifier, basic memory information, remaining memory capacity, node memory status, and borrowed memory status;

[0163] The recording subunit is used to record the memory resource information of each server node into the memory resource table.

[0164] In some embodiments, a subunit is constructed to send a first timestamp to a target server node; wherein the target server node is any one of all server nodes; after sending the first timestamp, a test packet and its message digest are sent to the target server node; after sending the test packet, a second timestamp is sent to the target server node; a target transmission time is received from the target server node; wherein the target transmission time is the difference between the second timestamp and the first timestamp received by the target server node; and the mutual transmission speed between the master node and the target server node is determined based on the data length of the test packet and the target transmission time.

[0165] In some embodiments, it further includes a hash operation unit, a judgment unit, and a feedback unit;

[0166] The hash operation unit is used to perform hash operations on test packets received from other server nodes to obtain hash values;

[0167] The judgment unit is used to determine whether the hash value is consistent with the message digest carried by the test packet;

[0168] The feedback unit is used to provide feedback on the transmission time to other server nodes when the hash value matches the message digest carried in the test packet; where the transmission time is the difference between two timestamps transmitted by other server nodes.

[0169] In some embodiments, the determining unit includes a feature determining subunit, an obtaining subunit, a sorting subunit, and an as-subunit;

[0170] The feature determination subunit is used to determine the node feature data of each server node based on the memory resource table and the node inter-transmission speed table.

[0171] The sub-units are obtained, which are used to input the node feature data of each server node into the graph neural network model to obtain the probability value of each server node except for the task node;

[0172] The sorting subunit is used to determine the node lending priority sorting according to the memory capacity carried in the memory expansion request, the probability value of each server node, and the remaining memory capacity.

[0173] As a sub-unit, it is used to select the first server node in the node lending priority sort as the lending node.

[0174] In some embodiments, the sorting subunit is used to sort the server nodes in descending order according to their probability values ​​and delete server nodes whose remaining memory capacity is less than the memory capacity carried in the memory expansion request, so as to obtain the node lending priority sort.

[0175] In some embodiments, the sub-unit is further configured to, upon receiving a rejection signal sent by the lending node, designate the next server node in the node lending priority ranking as the new lending node.

[0176] In some embodiments, a prompting unit is also included;

[0177] The prompting unit is used to send a prompt message to the task node when the remaining memory capacity of the first server node in the node lending priority sort is less than the memory capacity carried in the memory expansion request, so that the task node can send the memory expansion request again when memory is insufficient.

[0178] In some embodiments, after sending the node information of the lending node to the task node, the system further includes an accumulation unit, a sample update unit, and a training unit.

[0179] An accumulation unit is used to increment the memory allocation count by one each time a memory allocation is completed; initially, the memory allocation count is zero.

[0180] The sample update unit is used to update the sample data using the node feature data and label data corresponding to the number of memory allocations when the number of memory allocations reaches a set threshold.

[0181] The training unit is used to retrain the graph neural network model using the updated sample data.

[0182] In some embodiments, the system further includes an address construction unit, a traversal unit, a first recording unit, a second recording unit, and an address sending unit;

[0183] The address construction unit is used to construct a memory address table when it is a lending node; the memory address table includes the starting address of the segment, the segment length, and the total length; the initial total length is zero.

[0184] The traversal unit is used to traverse all available memory in the memory status information and obtain the current available memory length when a memory borrowing request is received from a task node. The memory status information includes the starting address of each free segment, its segment length, and the borrowable flag.

[0185] The first recording unit is used to record the current available memory length and its corresponding starting address to the memory address table, and update the total length and memory status information, when the sum of the current available memory length and the total length is less than the memory capacity carried in the memory expansion request.

[0186] The second recording unit is used to record the starting address corresponding to the current available memory length into the memory address table when the sum of the current available memory length and the total length is greater than or equal to the memory capacity carried in the memory expansion request; record the difference between the memory capacity carried in the memory expansion request and the total length as the segment length into the memory address table; and update the total length and memory status information.

[0187] The address sending unit is used to send the memory address table to the task node when the total length is equal to the memory capacity carried in the memory expansion request.

[0188] In some embodiments, it further includes a memory update unit and a task processing unit;

[0189] The memory update unit is used to update the expandable memory mapping table based on the memory address table sent by the lending node when it is a task node; the expandable memory mapping table includes the lending node identifier, the fragment starting address, and the fragment length;

[0190] The task processing unit is used to send the task to be processed to the target lending node according to the expandable memory mapping table when the task to be processed calls virtual memory, so that the target lending node can decode and process the task to be processed.

[0191] For a description of the features in the embodiment corresponding to the server memory expansion device, please refer to the relevant description in the embodiment corresponding to the server memory expansion method, which will not be repeated here.

[0192] As can be seen from the above technical solution, an initial memory resource management mechanism is established, and probe signals are sent to each server node. The memory resources of each server node and the speed of inter-server data transfer are important factors affecting memory allocation. Therefore, in this application, a memory resource table is generated based on the memory resource information fed back by each server node; and a node inter-server data transfer speed table is constructed based on the transmission time of data packets between nodes. Upon receiving a memory expansion request from a task node, the lending node is determined based on the memory resource table, the node inter-server data transfer speed table, and the memory capacity carried in the memory expansion request. The node information of the lending node is then sent to the task node so that the task node can access the memory of the lending node based on the node information. When the lending node successfully lends memory to the task node, its node memory status information changes. The lending node feeds back its memory status information to the master node, and the master node updates the memory resource table based on the node memory status information fed back by the lending node, thereby ensuring the accuracy of the memory resource table. In this technical solution, each server node, through the initialization of the memory resource management mechanism, ensures normal communication with other server nodes, enabling adaptive and efficient multi-node interconnection. By maintaining a memory resource table and a node transfer speed table, the master node can achieve efficient integration and dynamic management of memory resources among server nodes, solving the current challenges of memory expansion and high-speed access. The memory expansion method provided in this application significantly enhances the computing power of server nodes, providing strong support for fields such as large-scale models, high-performance computing, and game development.

[0193] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above-described server memory expansion method embodiments.

Claims

1. A method for server memory extension, the method comprising: The method suitable for a master node comprises: initializing a memory resource management mechanism and sending a probe signal to each server node; generating a memory resource table according to memory resource information fed back by each server node; constructing a node mutual transmission speed table according to transmission time of data packets between nodes; in the case of receiving a memory expansion request sent by a task node, determining a lending node that lends memory to the task node according to the memory resource table, the node mutual transmission speed table and memory capacity carried in the memory expansion request, and sending node information of the lending node to the task node; updating the memory resource table according to node memory state information fed back by the lending node; constructing a node mutual transmission speed table according to transmission time of data packets between nodes, comprising: sending a first timestamp to a target server node; wherein the target server node is any one of all server nodes; after completing sending of the first timestamp, sending a test packet and a message digest thereof to the target server node; after completing sending of the test packet, sending a second timestamp to the target server node; receiving a target transmission time fed back by the target server node; wherein the target transmission time is a difference between the second timestamp and the first timestamp received by the target server node; determining mutual transmission speed between the master node and the target server node based on data length of the test packet and the target transmission time.

2. The server memory extension method of claim 1, wherein, generating a memory resource table according to memory resource information fed back by each server node, comprising: receiving memory resource information fed back by each server node; constructing a memory resource table in a set data format; wherein the data format comprises a combination of node identification, memory basic information, memory remaining capacity, node memory state and lending memory state; recording memory resource information of each server node to the memory resource table.

3. The server memory extension method of claim 1 or 2, wherein, determining a lending node that lends memory to the task node according to the memory resource table, the node mutual transmission speed table and memory capacity carried in the memory expansion request, comprising: determining node characteristic data of each server node according to the memory resource table and the node mutual transmission speed table; inputting the node characteristic data of each server node into a graph neural network model to obtain a probability value of each server node except the task node; determining a node lending priority ranking according to memory capacity carried in the memory expansion request, the probability value of each server node and memory remaining capacity; taking a first server node in the node lending priority ranking as the lending node.

4. The server memory extension method of claim 3, wherein, determining a node lending priority ranking according to memory capacity carried in the memory expansion request, the probability value of each server node and memory remaining capacity, comprising: arranging each server node in descending order according to the probability value, and deleting a server node with memory remaining capacity less than the memory capacity carried in the memory expansion request to obtain the node lending priority ranking.

5. The server memory extension method of claim 3, wherein, After the first server node in the node lending priority ranking is taken as the lending node, the method further comprises: In the case that a rejection signal sent by the lending node is received, the next server node in the node lending priority ranking is taken as a new lending node.

6. The server memory extension method of claim 3, wherein, After the node information of the lending node is sent to the task node, the method further comprises: The memory allocation times are increased by one each time the memory is allocated; and the memory allocation times are zero in the initial state; In the case that the memory allocation times reach a set number threshold, the node feature data corresponding to the memory allocation times and the label data are used to update the sample data; The graph neural network model is retrained using the updated sample data.

7. The server memory extension method of claim 1, wherein, The method further comprises: In the case that the server itself is a lending node, a memory address table is constructed; the memory address table comprises a segment start address, a segment length, and a total length; and the total length is zero in the initial state; In the case that a memory lending request sent by a task node is received, all lendable memories in the memory state information are traversed to obtain a current lendable memory length; the memory state information comprises a start address, a segment length, and a lendable flag of each idle segment; In the case that the sum of the current lendable memory length and the total length is less than the memory capacity carried in the memory expansion request, the current lendable memory length and the start address corresponding thereto are recorded to the memory address table, and the total length and the memory state information are updated; In the case that the sum of the current lendable memory length and the total length is greater than or equal to the memory capacity carried in the memory expansion request, the start address corresponding to the current lendable memory length is recorded to the memory address table, the difference between the memory capacity carried in the memory expansion request and the total length is recorded to the memory address table as a segment length, and the total length and the memory state information are updated; In the case that the total length is equal to the memory capacity carried in the memory expansion request, the memory address table is sent to the task node.

8. The server memory extension method of claim 1, wherein, The method further comprises: In the case that the server itself is a task node, an extendable memory mapping table is updated according to the memory address table sent by the lending node; the extendable memory mapping table comprises a lending node identifier, a segment start address, and a segment length; In the case that a virtual memory is called by a to-be-processed task, the to-be-processed task is sent to a target lending node according to the extendable memory mapping table, so that the target lending node decodes and processes the to-be-processed task.

9. An electronic device, comprising: The method comprises: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the server memory expansion method according to any one of claims 1 to 8.

Images