Server memory expansion method and electronic equipment

Abstract

The invention discloses a server memory expansion method and electronic equipment, relates to the technical field of servers, and comprises the following steps: initializing a memory resource management mechanism, and sending a detection signal to each server node; generating a memory resource table according to the memory resource information fed back by each server node; and constructing a node mutual transmission speed table according to the transmission time of the data packet between the nodes. The main node can determine the lending node by maintaining the memory resource table and the node mutual transmission speed table when the task node sends the memory expansion request, and send the node information of the lending node to the task node, so that the task node can call the memory of the lending node. And the main node updates the memory resource table according to the node memory state information fed back by the lending node, so that the accuracy of the memory resource table is ensured. By maintaining the memory resource table and the node mutual transmission speed table, efficient integration and dynamic management of memory resources can be realized, and the memory expansion capability and the data access speed are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of servers, and in particular to a server memory expansion method and electronic equipment. BACKGROUND

[0002] In today's era of rapid technological development, the rapid progress of large models, high-performance computing and game development is reshaping computing demands. The applications in these fields pose unprecedented challenges to computing power and storage requirements.

[0003] The training and inference of large models require vast memory resources, while high-performance computing requires extremely fast access speed when processing large amounts of data. At the same time, game development also has strict requirements on memory and bandwidth in real-time rendering and complex scenarios. In order to support these advanced applications, the memory expansion capability of traditional server architecture often appears to be insufficient, and it often cannot provide sufficient speed when facing large-scale data transmission.

[0004] Therefore, how to improve the memory expansion capability and provide faster data access speed is a problem to be solved by those skilled in the art. SUMMARY

[0005] The present application provides a server memory expansion method and electronic equipment to at least solve the problem of insufficient memory expansion capability in related technologies, which cannot provide sufficient speed.

[0006] The present application provides a server memory expansion method suitable for a master node, the method comprising: initializing a memory resource management mechanism and sending a probe signal to each server node; generating a memory resource table according to memory resource information fed back by each server node; constructing a node mutual transmission speed table according to the transmission time of data packets between nodes; in the case of receiving a memory expansion request sent by a task node, determining a lending node that lends memory to the task node according to the memory resource table, the node mutual transmission speed table and the memory capacity carried in the memory expansion request, and sending node information of the lending node to the task node; updating the memory resource table according to node memory state information fed back by the lending node.

[0007] The present application also provides a server memory expansion device suitable for a master node, the device comprising an initialization unit, a generating unit, a constructing unit, a determining unit, a sending unit and an updating unit; The initialization unit is configured to initialize a memory resource management mechanism and send a probe signal to each server node; The generating unit is configured to generate a memory resource table according to the memory resource information fed back by each server node; The constructing unit is configured to construct a node mutual transmission speed table according to the transmission time of the data packet between the nodes; The determining unit is configured to, in a case where the memory expansion request sent by the task node is received, determine a lending node that lends memory to the task node according to the memory resource table, the node mutual transmission speed table, and the memory capacity carried in the memory expansion request. The sending unit is configured to send the node information of the lending node to the task node. The updating unit is configured to update the memory resource table according to the node memory state information fed back by the lending node.

[0008] The application further provides an electronic device, including a memory configured to store a computer program, and a processor configured to execute the computer program to implement the steps of any one of the server memory expansion methods.

[0009] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of any one of the server memory expansion methods.

[0010] The application further provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the steps of any one of the server memory expansion methods.

[0011] By the present application, the memory resource management mechanism is initialized, and a probe signal is sent to each server node. The memory resources of each server node and the inter-transmission speed between server nodes are important factors affecting memory invocation, so in the present application, a memory resource table is generated according to the memory resource information fed back by each server node; a node inter-transmission speed table is constructed according to the transmission time of data packets between nodes. In the case of receiving a memory expansion request sent by a task node, according to the memory resource table, the node inter-transmission speed table and the memory capacity carried in the memory expansion request, the lending node that lends memory to the task node is determined, and the node information of the lending node is sent to the task node, so that the task node can invoke the memory of the lending node according to the node information. When the lending node successfully lends memory to the task node, the node memory state information of the lending node will change, and the lending node will feed back the memory state information to the master node, and the master node updates the memory resource table according to the node memory state information fed back by the lending node, so as to ensure the accuracy of the memory resource table. In the technical scheme, each server node initializes the memory resource management mechanism to ensure normal communication with other server nodes, and adaptive and efficient multi-node interconnection can be realized. The master node can realize efficient integration and dynamic management of memory resources between server nodes by maintaining the memory resource table and the node inter-transmission speed table, and solve the challenges of memory expansion and high-speed access currently faced. The memory expansion method provided in the present application greatly improves the computing capacity of the server node, and provides strong support for the fields of large models, high-performance computing and game development. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0013] Figure 1 A networking architecture diagram of server memory expansion provided for an embodiment of the present application; Figure 2 A flowchart of a server memory expansion method provided for an embodiment of the present application; Figure 3 A schematic diagram of a training set provided for an embodiment of the present application; Figure 4 A flowchart of a lending memory method provided for an embodiment of the present application; Figure 5 A networking architecture diagram of a coherent interconnection expansion unit provided for an embodiment of the present application; Figure 6 A schematic diagram of updating an extensible memory mapping table provided for an embodiment of the present application; Figure 7 A structural schematic diagram of a server memory expansion device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0014] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0015] It should be noted that in the description of the present application, the terms "comprise", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. The terms "first", "second" and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0016] In the rapid development of artificial intelligence today, the rise of large models has attracted widespread attention and research. Large models are favored for their strong language understanding and generation capabilities, however, their training and inference processes have extremely demanding requirements for computing resources, especially in terms of memory expansion and high-speed access. This challenge not only affects the performance of the model, but also raises new requirements for the design and implementation of the entire technical ecosystem.

[0017] Firstly, large models usually contain tens of billions or even hundreds of billions of parameters, which means they need to occupy a large amount of memory during runtime. As the model size continues to expand, traditional hardware architectures have been unable to meet their memory requirements. How to achieve efficient model training and inference under limited hardware conditions is a challenge we are currently facing. To address this challenge, new storage architectures are being explored, such as using distributed computing and memory sharing technology to achieve higher memory utilization and faster data access speeds. Secondly, the high-speed access of memory is one of the key factors for the efficient operation of large models. The training and inference process of large models often requires frequent reading and writing of data, which puts strict requirements on the bandwidth and latency of memory. However, existing storage technologies often cannot provide sufficient speed when faced with large-scale data transmission. This has led researchers to explore new generations of storage media, such as Non-Volatile Memory (NVM) and Three-Dimensional NAND Flash Memory (3D NAND), in order to achieve faster access speeds while maintaining data persistence.

[0018] In addition, as the application scenarios of large models continue to expand, such as natural language processing, computer vision, and speech recognition, how to achieve efficient inference on different devices has also become a pressing problem. In the context of the increasing popularity of edge computing and mobile devices, how to compress and optimize large models so that they can run efficiently in resource-constrained environments has become a focus of attention.

[0019] The current Compute Express Link (CXL) technology provides new possibilities for server memory expansion. CXL is a high-bandwidth, low-latency interconnection standard that can effectively connect central processing units (CPUs), accelerators, and memory devices. Combined with memory pooling technology, CXL not only integrates the memory resources of multiple servers into a shared pool, but also dynamically allocates memory to improve resource utilization. The implementation of this technology enables servers to flexibly allocate memory when faced with large models and high-performance computing tasks, significantly improving performance.

[0020] Therefore, the embodiment of the application provides a server memory expansion method, device, electronic equipment and storage medium. Each server node is deployed with a CXL expansion unit. The server node initializes a memory resource management mechanism to ensure normal communication with other server nodes, and can realize adaptive and efficient multi-node interconnection. The master node can realize efficient integration and dynamic management of memory resources between server nodes by maintaining a memory resource table and a node intercommunication speed table, solve the challenges of memory expansion and high-speed access, improve the computing capacity of the server node, and provide strong support for large models, high-performance computing and game development fields. Through memory pooling, the allocation of computing resources is more flexible, and users can dynamically adjust the memory according to actual needs, thereby solving the problems of memory expansion and access speed and meeting the complex needs of modern applications.

[0021] Figure 1 A networking architecture diagram of a server memory expansion provided by the embodiment of the application is provided. Each server node includes a hard disk, a memory, a central processing unit and a coherent interconnection expansion unit. The coherent interconnection expansion unit is a software module deployed in the server node. The coherent interconnection expansion unit of each server node realizes interconnection and intercommunication between server nodes through a high-speed coherent network (Remote Direct Memory Access, RDMA).

[0022] In order for those skilled in the art to better understand the application scheme, the application will be further described in detail below in combination with the drawings and specific embodiments.

[0023] Figure 2 A flowchart of a server memory expansion method provided by the embodiment of the application is provided, which is applicable to a master node. The method comprises the following steps. S201: Initialize a memory resource management mechanism and send a probe signal to each server node.

[0024] The server memory expansion scheme provided by the embodiment of the application is applicable to a system comprising a plurality of server nodes, and the server node can be referred to as a node. In order to realize memory management, a server node can be marked as an intranet master server node, referred to as a master node. The purpose of the master node is to build an intranet memory management system, and it is not required that all programs run on the master node. The master node can be set and can be transferred.

[0025] During server startup or maintenance, each server node initializes a memory resource management mechanism through a built-in coherent interconnection interface (CXL interface) and configures network parameters of the server node.

[0026] Taking the master node as an example, the master node configures network parameters through a built-in consistent interconnection interface, so as to realize communication with other server nodes through the network parameters; wherein, the network parameters can include network topology, node address and communication protocol.

[0027] S202: generating a memory resource table according to the memory resource information fed back by each server node.

[0028] In order to facilitate the master node to understand the memory resource situation of each server node, the master node can maintain a memory resource table.

[0029] In specific implementation, the master node can send a probe signal to other server nodes in the network through a CXL interface, and the probe signal contains identification information of the master node, so as to facilitate other server nodes to identify and respond. After receiving the probe signal, the other server nodes can feed back their own memory resource information to the master node according to a preset response strategy.

[0030] The preset response strategy contains information required by the memory resource information, which can include combinations of node identification, memory type, memory capacity, memory bandwidth, memory remaining capacity, node memory state and borrowed memory state.

[0031] The master node can receive the memory resource information fed back by each server node; construct a memory resource table according to a set data format; and record the memory resource information of each server node to the memory resource table.

[0032] Wherein, the data format can include combinations of node identification, memory basic information, memory remaining capacity, node memory state and borrowed memory state. The memory basic information can include memory type, memory capacity and memory bandwidth, etc.

[0033] Wherein, the node identification can be a node address, and the node address can adopt a node Internet Protocol (IP) address. The memory type can be Double Data Rate Fourth Generation SDRAM (DDR4) or Double Data Rate Fifth Generation SDRAM (DDR5), etc.

[0034] In order to intuitively and quickly distinguish different server nodes, a node serial number can be set for each server node in the memory resource table, for example, there are totally 10 server nodes, and the node serial numbers are 1 to 10 in turn.

[0035] The memory resource table constructed by the master node can be stored in the CXL expansion unit. Table 1 is a memory resource table provided in an embodiment of the present application.

[0036] Table 1 Memory resource table Table 1 is only an example of the data types contained in each row of data. In actual applications, the system includes multiple server nodes, and the memory resource table records the resource information of each of the server nodes.

[0037] The node memory state in the memory resource table is synchronized with the CXL memory state of each server node. The borrowed memory state of each server node records the memory capacity borrowed by the node from other nodes.

[0038] The memory state of each server node can be initialized in the form of a memory state table, which stores the memory state information of all free memory segments of the current node, for local or remote CXL memory invocation.

[0039] In specific implementations, each server node can maintain a node memory state table through the memory manager of the CXL expansion unit, which records the memory state information of the node. The memory state information can include the first address of a free memory segment, the segment capacity, and a segment lending flag. The segment lending flag indicates whether the free memory segment can be lent to other nodes. The memory state table of each server node is as follows: Table 2 Node memory state table The node memory state table includes the first address of all free memory segments of the node, the segment capacity, and the lending flag. Table 2 is an example of n free memory segments. The value of n is determined by the number of free memory segments in the server node. The numbers in Table 2 are only used to distinguish different memory segments and do not have other limitations.

[0040] The segment lending flag can have two values. When the segment lending flag is 1, it indicates that the free memory segment can be lent to other nodes for use. When the segment lending flag is 0, it indicates that the free memory segment cannot be lent to other nodes for use.

[0041] Since memory fragmentation affects program running efficiency, in actual applications, when the segment capacity is greater than 1G, the lending flag can be set to 1; when the segment capacity is less than or equal to 1G, the lending flag can be set to 0.

[0042] The total capacity of free memory in Table 2 is the sum of the capacities of all free memory segments. The total capacity of lent free memory is the sum of the capacities of all free memory segments with a lending flag of 1.

[0043] Since the memory of the node will exist the process of occupation and release, the node memory state table maintained by each node will change, when the memory state information in the node memory state table changes, the node can transmit the memory state information to the master node, so that the master node updates the memory resource table in time.

[0044] The borrowed memory state records the memory capacity borrowed by each server node to other nodes, in order to facilitate memory management, each server node can maintain a borrowed memory state table, which includes the IP and capacity of the borrowed memory, as shown in Table 3 below is the borrowed memory state table of a single server node.

[0045] Table 3 borrowed memory state table .

[0046] S203: According to the transmission time of the data packet between nodes, a node mutual transmission speed table is constructed.

[0047] Considering that the time between different nodes may not be completely synchronized in actual application, in order to ensure the accuracy of the data packet transmission time between nodes, the master node can use the method of transmitting time stamp, test packet and time stamp to other nodes in turn to determine the transmission time of the test packet.

[0048] Taking any one of all server nodes, i.e. the target server node, as an example, the master node can send the first time stamp to the target server node; after completing the sending of the first time stamp, the test packet and its message digest are sent to the target server node. After completing the sending of the test packet, the second time stamp is sent to the target server node. The target transmission time fed back by the target server node is received; wherein the target transmission time is the difference between the second time stamp and the first time stamp received by the target server node. Based on the data length of the test packet and the target transmission time, the mutual transmission speed between the master node and the target server node is determined.

[0049] Referring to the way of determining the mutual transmission speed by the master node transmitting time stamp, test packet and time stamp to the target server node in turn, each server node can use the way of transmitting time stamp, test packet and time stamp to other server nodes in turn to determine the mutual transmission speed between nodes.

[0050] When the master node receives the test packet sent by other server nodes, the hash value is obtained by performing hash operation on the test packet. It is judged whether the hash value is consistent with the message digest carried by the test packet. In the case that the hash value is consistent with the message digest carried by the test packet, it means that the data transmission is complete and consistent, at this time the master node can feed back the transmission time to other server nodes; wherein the transmission time is the difference between the two time stamps transmitted by other server nodes.

[0051] In a specific implementation, the master node can send each of the remaining nodes a verification data packet and its message digest. The verification data packet can be customized by the user and is usually about 1 GB in size. The message digest algorithm (Message-Digest Algorithm 5, MD5) is used to encode the verification data packet to obtain the message digest, i.e., MD5 encoding. The receiving node verifies the correctness of the message digest. Thereafter, each node is traversed, and the data packet is sequentially sent to the remaining nodes to complete the transmission of the node-to-node transmission speed table.

[0052] Taking any two server nodes, such as node A and node B, as an example, it is considered that the local times of node A and node B can not be unified, and therefore the same node sends the time stamp. Taking the transmission speed of the test node A to the node B as an example, node A first sends the time stamp t1 to node B, and then sends the test packet. After completing the sending of the test packet, the time stamp t2 is sent. Thereafter, the node B locally verifies whether the MD5 of the test packet is correct. If not, repeat the step; if correct, the node B calculates the transmission time, i.e., t2-t1, and feeds back the transmission time to the master node. The quotient of the data length of the test packet and the transmission time is the transmission speed, and the master node fills the transmission speed into the corresponding position of the node-to-node transmission speed table.

[0053] In order to facilitate the management of the transmission time of the data packet between the nodes, the master node can construct a node-to-node transmission speed table, as shown in Table 4. The node-to-node transmission speed table is stored in the memory.

[0054] Table 4 Node-to-node transmission speed table In Table 4, node 1, node 2, and node 3 are node serial numbers, corresponding to the node serial numbers in the memory resource table. The i-th row and j-th column in Table 4 represent the transmission speed of server node i to server node j. For example, V 2,1 in Table 4 represents the transmission speed of node 2 to node 1. The position with a diagonal line in Table 4 represents that the transmission speed does not need to be calculated.

[0055] S204: In the case that the memory expansion request sent by the task node is received, the lending node that lends the memory to the task node is determined according to the memory resource table, the node-to-node transmission speed table, and the memory capacity carried in the memory expansion request, and the node information of the lending node is sent to the task node.

[0056] In the embodiment of the present application, the node characteristic data of each server node can be determined according to the memory resource table and the node mutual transmission speed table. The node characteristic data of each server node is input into the graph neural network model to obtain the probability value of each server node except the task node. The node lending priority ranking is determined according to the memory capacity carried in the memory expansion request, the probability value of each server node, and the memory remaining capacity. The first server node in the node lending priority ranking is taken as the lending node.

[0057] The node characteristic data can include upload speed, download speed, memory remaining capacity, lent memory capacity, and total amount of lent memory of the task node.

[0058] After obtaining the probability value of each server node, the server nodes can be arranged in descending order according to the probability value, and the server nodes with a memory remaining capacity less than the memory capacity carried in the memory expansion request are deleted to obtain the node lending priority ranking.

[0059] The higher the probability value of the server node, the more suitable the memory of the server node is for lending to the task node. Therefore, after arranging the server nodes in descending order according to the probability value, the first server node in the node lending priority ranking can be taken as the lending node.

[0060] S205: updating the memory resource table according to the node memory state information fed back by the lending node.

[0061] After receiving the node information of the lending node, the task node can send a memory lending request to the lending node. The lending node verifies whether the memory state of the local CXL expansion unit meets the demand. For example, when the total capacity of the lent free memory is greater than the memory capacity carried in the memory lending request, it is indicated that the memory state meets the demand.

[0062] When the memory state of the lending node meets the demand, the lending node determines the memory segment that can be lent to the task node according to the memory capacity required by the task node, and feeds back the start address and segment length of the memory segment to the task node, so that the task node can successfully call the memory of the lending node.

[0063] Since the lending node lends its memory to the task node, the node memory state information of the lending node changes. At this time, the lending node feeds back the node memory state information to the master node. The master node updates the memory resource table according to the memory state information, ensures the timeliness of updating the information in the memory resource table, and guarantees the accuracy of the memory resource table.

[0064] It can be seen from the above technical solution that the memory resource management mechanism is initialized, and a probe signal is sent to each server node. The memory resources of each server node and the mutual transmission speed between the server nodes are important factors affecting memory calling. Therefore, in the present application, a memory resource table is generated according to the memory resource information fed back by each server node, and a node mutual transmission speed table is constructed according to the transmission time of the data packets between the nodes. In the case of receiving a memory expansion request sent by a task node, the lending node that lends memory to the task node is determined according to the memory resource table, the node mutual transmission speed table, and the memory capacity carried in the memory expansion request, and the node information of the lending node is sent to the task node, so that the task node can call the memory of the lending node according to the node information. When the lending node successfully lends memory to the task node, the node memory state information of the lending node will change, and the lending node will feed back the memory state information to the master node. The master node updates the memory resource table according to the node memory state information fed back by the lending node, thereby ensuring the accuracy of the memory resource table. In the technical solution, each server node initializes the memory resource management mechanism to ensure normal communication with other server nodes, and adaptive and efficient multi-node interconnection can be achieved. The master node can realize efficient integration and dynamic management of memory resources between server nodes by maintaining the memory resource table and the node mutual transmission speed table, thereby solving the challenges of memory expansion and high-speed access. The memory expansion method provided in the present application greatly improves the computing capacity of the server nodes, and provides strong support for large models, high-performance computing, and game development.

[0065] In actual applications, after the first server node in the node lending priority ranking is taken as the lending node, there may be a case that the memory state of the lending node does not meet the lending demand. At this time, the lending node sends a rejection signal to the master node.

[0066] The master node takes the next server node in the node lending priority ranking as a new lending node when receiving the rejection signal sent by the lending node. The next server node is the next server node adjacent to the lending node.

[0067] When the memory capacity required by the task node for external lending is large, there may be a case that the memory capacity exceeds the memory remaining capacity of a single lending node. In this case, the memory of a single lending node can be allocated to the task node first, and a prompt information is sent to the task node, so that the task node can send a memory expansion request again in the case of insufficient memory.

[0068] In the specific implementation, after the first server node in the node lending priority ranking is taken as the lending node, it can be judged whether the memory remaining capacity of the first server node in the node lending priority ranking is less than the memory capacity carried in the memory expansion request.

[0069] In a case where the remaining memory capacity of the first server node in the node lending priority ranking is less than the memory capacity carried in the memory expansion request, it is indicated that the lent memory in this allocation cannot meet the memory demand of the task node. At this time, the master node can send a prompt to the task node so that the task node sends a memory expansion request again in the case of insufficient memory.

[0070] In the embodiments of the present application, the lent memory is flexibly allocated to the task node according to the lending situation of the lending node. In a case where the memory state of the lending node does not meet the lending demand, the next server node in the node lending priority ranking can be used as a new lending node. In a case where the remaining memory capacity of the lending node is less than the memory capacity required by the task node, the lending node can be allocated to the task node multiple times to meet the memory capacity required by the task node to execute the task.

[0071] In the embodiments of the present application, the graph neural network model can be used to analyze the node feature data of each server node to determine the probability value of each server node.

[0072] Before using the graph neural network model, the graph neural network model needs to be trained. In a specific implementation, the graph neural network model memory allocator of the CXL expansion unit of the master node records all historical memory allocation data to construct a training set of the graph neural network model.

[0073] Figure 3 A schematic diagram of a training set provided by the embodiments of the present application, the training set can include two parts of input data and label data, wherein the input data and the node feature data are constructed in the same way. The input data contains a plurality of samples, Figure 3 The K samples are taken as an example. Each sample contains n node training input data and its training label data. The training input data includes upload speed, download speed, remaining memory capacity, total lent memory, and total lent memory of the task node. The training label data is used to represent whether the allocation is ultimately successful. Allocation success is recorded as 1, failure is recorded as -1, and the node is not allocated as 0.

[0074] The training input data and the training label data contained in the training set are used to train the graph neural network model; the graph neural network model parameters are optimized according to the gradient descent algorithm, and the weights of the graph neural network model are output to obtain the trained graph neural network model.

[0075] In order to improve the fit of the graph neural network model with the current application scenario, a memory allocation number D can be set. Each time D new memory allocations are added, the graph neural network model is retrained once to update the weights of the graph neural network model.

[0076] The master node determines a lending node each time, which is equivalent to completing a memory allocation.

[0077] In a specific implementation, after the node information of the lending node is sent to the task node, the number of memory allocations is increased by one each time a memory allocation is completed; and the number of memory allocations is zero in an initial state. In a case where the number of memory allocations reaches a set number threshold, sample data is updated by using node feature data and label data corresponding to the number of memory allocations; and the graph neural network model is retrained by using the updated sample data.

[0078] The number threshold is the number of memory allocations D, and the value of D can be flexibly set based on actual requirements, which is not limited herein.

[0079] The sample data is updated by using node feature data and label data corresponding to the number of memory allocations, which can be deleting the first D data in the original sample data, adding the node feature data and label data corresponding to the D memory allocations to the original sample data, and thus obtaining the updated sample data.

[0080] In the embodiments of the present application, the sample data is updated based on the historical memory allocation, and the graph neural network model is trained by using the updated sample data, which ensures that the graph neural network model is more suitable for the current application scenario, so that the output node lending priority ranking can better meet the scene requirements of the current application scenario, and the accuracy of memory resource allocation is improved.

[0081] Figure 4 A flowchart of a method for lending memory provided in the embodiments of the present application, the method comprising: S401: In a case where the server node itself belongs to a lending node, constructing a memory address table.

[0082] In actual application, any one server node can be a task node or a lending node.

[0083] Regardless of which server node is a lending node, the processing mode of lending memory is similar. Taking the case where the master node belongs to a lending node as an example, when the master node determines itself as a lending node according to the memory resource table, the node intercommunication speed table and the memory capacity carried in the memory expansion request, in order to facilitate the feedback of the memory segment that can be lent to the task node, a memory address table can be constructed first.

[0084] The memory address table can include a segment first address, a segment length and a total length; and the memory address table has no specific data in an initial state, and the total length is zero. Table 5 shows a memory address table provided in the embodiments of the present application.

[0085] Table 5 Memory address table .

[0086] S402: In the case of receiving the memory borrowing request sent by the task node, all the available memories in the memory state information are traversed to obtain the current available memory length.

[0087] The memory state information includes the first address of each free segment, the segment length and the available flag.

[0088] The node can record the memory state information through the node memory state table. In combination with the node memory state table shown in Table 2, it can be known that all the free memory segments of the node with the available flag being 1.

[0089] Generally, the data in each row of the node memory state table can be arranged in descending order from high to low according to the capacity of the free memory segment. In actual application, the memory state table can be traversed, and the first free memory segment is obtained in the initial state. The segment capacity corresponding to the first free memory segment is the current available memory length. When the segment capacity of the first free memory segment does not meet the memory capacity required by the task node, the segment capacity of the first free memory segment is taken as the total length, and the segment capacity of the next free memory segment is read as the current available memory length. Until the total length reaches the memory capacity required by the task node, the operation is ended.

[0090] S403: In the case that the sum of the current available memory length and the total length is less than the memory capacity carried in the memory expansion request, the current available memory length and the corresponding first address are recorded to the memory address table, and the total length and the memory state information are updated.

[0091] The memory address table has no specific data in the initial state, and the total length is zero. Each time a segment first address and a segment length are added, the total length is updated. The total length is the sum of all the segment lengths recorded in the memory address table.

[0092] In the case that the sum of the current available memory length and the total length is less than the memory capacity carried in the memory expansion request, the current available memory length and the corresponding first address can be recorded to the memory address table, and the total length and the memory state information are updated.

[0093] Since the sum of the current available memory length and the total length is less than the memory capacity carried in the memory expansion request, it indicates that the currently borrowed memory cannot meet the memory requirement of the task node. At this time, the next data in the memory state table is read, and the segment capacity corresponding to the free memory segment in the next data is the current available memory length. By analogy, until the sum of the current available memory length and the total length is greater than or equal to the memory capacity carried in the memory expansion request, S404 is executed.

[0094] S404: In a case where the sum of the current lendable memory length and the total length is greater than or equal to the memory capacity carried in the memory expansion request, record the first address corresponding to the current lendable memory length to the memory address table, record the difference between the memory capacity carried in the memory expansion request and the total length as the segment length to the memory address table, and update the total length and the memory state information.

[0095] In a case where the sum of the current lendable memory length and the total length is greater than the memory capacity carried in the memory expansion request, it is indicated that all the segment capacities of the current idle memory segments do not need to be lent out, and only the difference between the memory capacity carried in the memory expansion request and the total length needs to be lent out.

[0096] In a specific implementation, the first address corresponding to the current lendable memory length can be recorded to the memory address table, the difference between the memory capacity carried in the memory expansion request and the total length can be recorded as the segment length to the memory address table, and the total length and the memory state information can be updated.

[0097] S405: In a case where the total length is equal to the memory capacity carried in the memory expansion request, send the memory address table to the task node.

[0098] In a case where the total length is equal to the memory capacity carried in the memory expansion request, it is indicated that all the lent-out memories have been determined by the lending node, and the segment first addresses and the segment lengths corresponding to all the lent-out memories have been recorded in the memory address table. At this time, the lending node can send the memory address table to the task node, so that the task node can rely on the segment first addresses and the segment lengths recorded in the memory address table to implement the calling of the memory of the lending node.

[0099] In the embodiments of the present application, each server node relies on the built-in CXL expansion unit to implement the management of the memory state and the interaction between nodes. Based on the functions required to be implemented by the CXL expansion unit, it can be divided into a memory manager, a virtual memory pool, a virtual memory control, and an address translation.

[0100] Figure 5 A networking architecture diagram of a consistent interconnection expansion unit provided in the embodiments of the present application is provided. The consistent interconnection expansion unit is installed on each server node in the network, and includes a memory manager, a virtual memory controller, a virtual memory pool, and an address translator. When the server node is a master node, the consistent interconnection expansion unit of the master node also maintains a memory resource table, a node intercommunication speed table, and a graph neural network model. The graph neural network model is equivalent to a memory allocator. Figure 5 The modules and tables in the dashed box are only the modules and tables possessed by the master node.

[0101] The consistency interconnection expansion unit is referred to as a CXL unit. When a node initiates a memory request during the running of a task, the operating system of the node interacts with the memory manager of the CXL unit, and by comparing with the memory state, the local memory is called preferentially, and the memory state is updated. Then, the virtual memory controller transmits the total capacity of the idle memory that can be lent in the memory state to the master node CXL unit, and updates the information of the corresponding node in the memory resource table.

[0102] When a certain memory request of a certain program of a task node is judged to be unable to be satisfied by the local memory after comparison with the memory state of the task node, the memory is applied to the cluster and virtualization calling is performed, and the operation steps are as follows: Step 1: The virtual memory controller of the CXL unit of the task node sends a memory expansion request to the master node CXL unit (hereinafter referred to as the master CXL unit) through the CXL.cache protocol, including the memory capacity that needs to be expanded.

[0103] Step 2: The master CXL unit calls the related information in the memory resource table and the node intercommunication speed table, and inputs the information and the memory capacity required by the task node into the graph neural network model for node resource allocation.

[0104] Step 3: The master CXL unit sends the IP address of the lending node to the task node, and the task node sends a memory lending request to the lending node. The lending node verifies whether the memory state of the local CXL unit meets the demand, for example, when the total capacity of the idle memory that can be lent of the lending node is greater than or equal to the memory capacity of the task node, it means that the memory demand of the task node is met. If the demand is not met, the lending node sends a rejection signal to the master node, and the master node outputs the next node as the lending node according to the node lending priority order, and feeds back the IP address of the next lending node to the task node, until a lending node that meets the memory demand of the task node is selected.

[0105] For step 2, when the graph neural network model is used for memory allocation, the following operation steps can be included: 2.1: Construct node feature data for each node, and the format of the feature data is shown in Table 6.

[0106] Table 6 Node feature data table .

[0107] wherein the upload speed is the speed of the task node transmitting the verification data packet to the lending node, extracted from the node mutual transmission speed table; the download speed is the speed of the lending node transmitting the verification data packet to the task node, extracted from the node mutual transmission speed table; the memory remaining capacity is the memory remaining capacity of the lending node, extracted from the memory resource table; the total lending memory amount is the memory capacity of the lending node that has been lent to other nodes, extracted from the "lending memory state" field of the memory resource table; and the total lending memory amount of the task node is the memory capacity of the lending node that has been lent to the current task node, extracted from the "lending memory state" field of the memory resource table.

[0108] 2.2: Input the node characteristic data of each node (including the node characteristic data of the task node) into the graph neural network model for prediction.

[0109] wherein the features of each feature node in the graph neural network model are initialized as the node characteristic data of step 2.1, and the edges between the feature nodes are initialized as the inverse of the sum of the upload speed and the download speed of the two server nodes.

[0110] 2.3: The graph neural network model is inferred to obtain the probability value of each node except the task node. The probability value is sorted in descending order, and the nodes with a memory remaining capacity less than the required memory capacity of the current task node are deleted to obtain a node lending priority ranking.

[0111] In actual application, if the maximum value of the memory remaining capacity of all nodes is less than the required memory capacity of the task node, the current memory requirement is updated as min{the maximum value of the memory remaining capacity of all nodes, the current memory requirement}. If the lent memory is not enough, the task node applies for memory again.

[0112] 2.4: The first node in the node lending priority ranking is used as the lending node.

[0113] In actual application, the master node can also belong to the task node. Taking the case that the master node belongs to the task node as an example, the master node can update the extendable memory mapping table according to the memory address table sent by the lending node; wherein the extendable memory mapping table includes the lending node identifier, the segment first address and the segment length. In the case that the to-be-processed task calls the virtual memory, the to-be-processed task is sent to the target lending node according to the extendable memory mapping table, so that the target lending node decodes and processes the to-be-processed task.

[0114] In specific implementation, after receiving the memory address table, the task node can update the extendable memory mapping table in the address translator, Figure 6An updating extensible memory mapping table provided in the embodiment of the present application, the CXL unit translates the external address into the local access address through the address memory table, the local CPU calls the virtual memory pool as the local memory, the difference lies in that the calling address is different (the address is sequentially arranged after the local memory access address according to the request order), the extensible memory mapping table exists in the address translator of the CXL unit, and is called when the virtual memory is accessed. Figure 6 The address range of the local memory of the task node is 0000-000B, and the address range of the extended memory of the task node is 000B-003B, for example, each record in the extensible memory mapping table contains the IP of the lending node, the segment start address and the segment length.

[0115] When the current task uses the virtual memory, the memory manager of the CXL unit sends the data packet to be processed to the memory of the lending node through the address translator.

[0116] The data packet format is shown in Table 7, and the "start address" field in the data packet represents the real memory address of the lending node.

[0117] Table 7 Data packet format table .

[0118] In the embodiment of the present application, the task node can realize direct calling of the lending memory by updating the memory address fed back by the lending node to the extensible memory mapping table of the task node, and the memory calling efficiency is improved.

[0119] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment.

[0120] Figure 7 A structure diagram of a server memory extension device provided in the embodiment of the present application, suitable for a master node, the device comprises an initialization unit 71, a generating unit 72, a building unit 73, a determining unit 74, a sending unit 75 and an updating unit 76; The initialization unit 71 is used for initializing the memory resource management mechanism and sending a detection signal to each server node; The generating unit 72 is used for generating a memory resource table according to the memory resource information fed back by each server node; The building unit 73 is used for building a node mutual transmission speed table according to the transmission time of the data packet between nodes; The determining unit 74 is configured to, in a case where the memory expansion request sent by the task node is received, determine a lending node lending memory to the task node according to the memory resource table, the node mutual transmission speed table and the memory capacity carried in the memory expansion request. The sending unit 75 is configured to send the node information of the lending node to the task node. The updating unit 76 is configured to update the memory resource table according to the node memory state information fed back by the lending node.

[0121] In some embodiments, the initializing unit is configured to configure network parameters through a built-in coherent interconnection interface, so as to facilitate communication with other server nodes through the network parameters; wherein the network parameters include network topology, node address and communication protocol.

[0122] In some embodiments, the generating unit includes a receiving subunit, a constructing subunit and a recording subunit. The receiving subunit is configured to receive the memory resource information fed back by each server node. The constructing subunit is configured to construct the memory resource table in a set data format; wherein the data format includes a combination of node identification, memory basic information, memory residual capacity, node memory state and lent memory state. The recording subunit is configured to record the memory resource information of each server node to the memory resource table.

[0123] In some embodiments, the constructing subunit is configured to send a first time stamp to a target server node; wherein the target server node is any one of all the server nodes; after the sending of the first time stamp is completed, send a test packet and a message digest thereof to the target server node; after the sending of the test packet is completed, send a second time stamp to the target server node; receive a target transmission time fed back by the target server node; wherein the target transmission time is a difference between the second time stamp and the first time stamp received by the target server node; determine the mutual transmission speed between the master node and the target server node based on the data length of the test packet and the target transmission time.

[0124] In some embodiments, further including a hash operation unit, a judging unit and a feedback unit. The hash operation unit is configured to, in a case where the test packet sent by the other server node is received, perform hash operation on the test packet to obtain a hash value. The judging unit is configured to judge whether the hash value is consistent with the message digest carried by the test packet. The feedback unit is configured to, in a case where the hash value is consistent with the message digest carried by the test packet, feed back a transmission time to the other server node; wherein the transmission time is a difference between two time stamps transmitted by the other server node.

[0125] In some embodiments, the determining unit comprises a feature determining subunit, a obtaining subunit, a ranking subunit and a serving subunit; The feature determining subunit is configured to determine the node feature data of each server node according to the memory resource table and the node intercommunication speed table; The obtaining subunit is configured to input the node feature data of each server node into the graph neural network model to obtain the probability value of each server node except the task node; The ranking subunit is configured to determine the node lending priority ranking according to the memory capacity carried in the memory expansion request, the probability value of each server node and the memory remaining capacity; The serving subunit is configured to serve the first server node in the node lending priority ranking as the lending node.

[0126] In some embodiments, the ranking subunit is configured to arrange the server nodes in descending order according to the probability value, and delete the server nodes with the memory remaining capacity less than the memory capacity carried in the memory expansion request, to obtain the node lending priority ranking.

[0127] In some embodiments, the serving subunit is further configured to serve the next server node in the node lending priority ranking as the new lending node in the case that the lending node sends a rejection signal.

[0128] In some embodiments, the method further comprises a prompting unit; The prompting unit is configured to send a prompt information to the task node in the case that the memory remaining capacity of the first server node in the node lending priority ranking is less than the memory capacity carried in the memory expansion request, so that the task node sends the memory expansion request again in the case of insufficient memory.

[0129] In some embodiments, after sending the node information of the lending node to the task node, the method further comprises an accumulating unit, a sample updating unit and a training unit; The accumulating unit is configured to add one to the memory allocation times each time the memory allocation is completed; wherein the memory allocation times is zero in the initial state; The sample updating unit is configured to update the sample data by using the node feature data corresponding to the memory allocation times and the label data in the case that the memory allocation times reaches a set number threshold; The training unit is configured to retrain the graph neural network model by using the updated sample data.

[0130] In some embodiments, the method further comprises an address constructing unit, a traversing unit, a first recording unit, a second recording unit and an address sending unit; An address construction unit is configured to construct a memory address table in a case where the server memory expansion device itself belongs to the lending node; wherein the memory address table comprises a segment first address, a segment length and a total length; and the total length is zero in an initial state; A traversal unit is configured to traverse all lendable memories in the memory state information to obtain a current lendable memory length in a case where a memory lending request sent by the task node is received; wherein the memory state information comprises a first address of each free segment, a segment length and a lendable flag of the free segment; A first recording unit is configured to record the current lendable memory length and a corresponding first address to the memory address table and update the total length and the memory state information in a case where a sum of the current lendable memory length and the total length is less than the memory capacity carried in the memory expansion request; A second recording unit is configured to record the first address corresponding to the current lendable memory length to the memory address table, record a difference between the memory capacity carried in the memory expansion request and the total length as the segment length to the memory address table, and update the total length and the memory state information in a case where the sum of the current lendable memory length and the total length is greater than or equal to the memory capacity carried in the memory expansion request; An address sending unit is configured to send the memory address table to the task node in a case where the total length is equal to the memory capacity carried in the memory expansion request.

[0131] In some embodiments, the server memory expansion device further comprises a memory updating unit and a task processing unit; The memory updating unit is configured to update the extendable memory mapping table according to the memory address table sent by the lending node in a case where the server memory expansion device itself belongs to the task node; wherein the extendable memory mapping table comprises a lending node identifier, a segment first address and a segment length; The task processing unit is configured to send the to-be-processed task to a target lending node for decoding and processing by the target lending node according to the extendable memory mapping table in a case where the to-be-processed task calls a virtual memory.

[0132] The features of the embodiments of the server memory expansion device can be referred to the related descriptions of the embodiments of the server memory expansion method, which will not be repeated here.

[0133] It can be seen from the above technical solution that the memory resource management mechanism is initialized, and a probe signal is sent to each server node. The memory resources of each server node and the mutual transmission speed between the server nodes are important factors affecting memory calling. Therefore, in the present application, a memory resource table is generated according to the memory resource information fed back by each server node, and a node mutual transmission speed table is constructed according to the transmission time of data packets between nodes. In the case of receiving a memory expansion request sent by a task node, the lending node that lends memory to the task node is determined according to the memory resource table, the node mutual transmission speed table and the memory capacity carried in the memory expansion request, and the node information of the lending node is sent to the task node, so that the task node can call the memory of the lending node according to the node information. When the lending node successfully lends memory to the task node, the node memory state information of the lending node will change, and the lending node will feed back the memory state information to the master node. The master node updates the memory resource table according to the node memory state information fed back by the lending node, thereby ensuring the accuracy of the memory resource table. In this technical solution, each server node initializes the memory resource management mechanism to ensure normal communication with other server nodes, and adaptive and efficient multi-node interconnection can be achieved. The master node can maintain the memory resource table and the node mutual transmission speed table to achieve efficient integration and dynamic management of memory resources between server nodes, and solve the challenges of memory expansion and high-speed access currently faced. The memory expansion method provided in the present application greatly improves the computing power of the server node, and provides strong support for the fields of large models, high-performance computing and game development.

[0134] Embodiments of the present application also provide an electronic device comprising a memory and a processor, the memory storing a computer program, and the processor being configured to run the computer program to perform the steps in any of the above server memory expansion method embodiments.

Claims

1. A method for server memory extension, the method comprising: The method suitable for a master node comprises: initializing a memory resource management mechanism and sending a probe signal to each server node; generating a memory resource table according to memory resource information fed back by each server node; constructing a node mutual transmission speed table according to transmission time of data packets between nodes; in the case of receiving a memory expansion request sent by a task node, determining a lending node that lends memory to the task node according to the memory resource table, the node mutual transmission speed table and memory capacity carried in the memory expansion request, and sending node information of the lending node to the task node; updating the memory resource table according to node memory state information fed back by the lending node.

2. The server memory extension method of claim 1, wherein, The method of generating a memory resource table according to memory resource information fed back by each server node comprises: receiving memory resource information fed back by each server node; constructing a memory resource table according to a set data format; wherein the data format comprises a combination of node identification, memory basic information, memory remaining capacity, node memory state and lent memory state; recording memory resource information of each server node to the memory resource table.

3. The server memory extension method of claim 1, wherein, The method of constructing a node mutual transmission speed table according to transmission time of data packets between nodes comprises: sending a first timestamp to a target server node; wherein the target server node is any one of all server nodes; after completing sending of the first timestamp, sending a test packet and a message digest thereof to the target server node; after completing sending of the test packet, sending a second timestamp to the target server node; receiving a target transmission time fed back by the target server node; wherein the target transmission time is a difference between the second timestamp and the first timestamp received by the target server node; determining mutual transmission speed between the master node and the target server node based on data length of the test packet and the target transmission time.

4. The server memory extension method according to any one of claims 1 to 3, wherein, The method of determining a lending node that lends memory to the task node according to the memory resource table, the node mutual transmission speed table and memory capacity carried in the memory expansion request comprises: determining node characteristic data of each server node according to the memory resource table and the node mutual transmission speed table; inputting the node characteristic data of each server node into a graph neural network model to obtain a probability value of each server node except the task node; determining a node lending priority ranking according to memory capacity carried in the memory expansion request, the probability value of each server node and memory remaining capacity; taking a first server node in the node lending priority ranking as the lending node.

5. The server memory extension method of claim 4, wherein, The method of determining a node lending priority ranking according to memory capacity carried in the memory expansion request, the probability value of each server node and memory remaining capacity comprises: arranging each server node in descending order according to the probability value, and deleting a server node with memory remaining capacity less than the memory capacity carried in the memory expansion request to obtain the node lending priority ranking.

6. The server memory extension method of claim 4, wherein, After the first server node in the node lending priority ranking is taken as the lending node, the method further comprises: In the case that a rejection signal sent by the lending node is received, the next server node in the node lending priority ranking is taken as a new lending node.

7. The server memory extension method of claim 4, wherein, After the node information of the lending node is sent to the task node, the method further comprises: The memory allocation times are increased by one each time the memory is allocated; and the memory allocation times are zero in the initial state; In the case that the memory allocation times reach a set number threshold, the node feature data corresponding to the memory allocation times and the label data are used to update the sample data; The graph neural network model is retrained using the updated sample data.

8. The server memory extension method of claim 1, wherein, The method further comprises: In the case that the server itself is a lending node, a memory address table is constructed; the memory address table comprises a segment start address, a segment length, and a total length; and the total length is zero in the initial state; In the case that a memory lending request sent by a task node is received, all lendable memories in the memory state information are traversed to obtain a current lendable memory length; the memory state information comprises a start address, a segment length, and a lendable flag of each idle segment; In the case that the sum of the current lendable memory length and the total length is less than the memory capacity carried in the memory expansion request, the current lendable memory length and the start address corresponding thereto are recorded to the memory address table, and the total length and the memory state information are updated; In the case that the sum of the current lendable memory length and the total length is greater than or equal to the memory capacity carried in the memory expansion request, the start address corresponding to the current lendable memory length is recorded to the memory address table, the difference between the memory capacity carried in the memory expansion request and the total length is recorded to the memory address table as a segment length, and the total length and the memory state information are updated; In the case that the total length is equal to the memory capacity carried in the memory expansion request, the memory address table is sent to the task node.

9. The server memory extension method of claim 1, wherein, The method further comprises: In the case that the server itself is a task node, an extendable memory mapping table is updated according to the memory address table sent by the lending node; the extendable memory mapping table comprises a lending node identifier, a segment start address, and a segment length; In the case that a virtual memory is called by a to-be-processed task, the to-be-processed task is sent to a target lending node according to the extendable memory mapping table, so that the target lending node decodes and processes the to-be-processed task.

10. An electronic device, comprising: The method comprises: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the server memory expansion method according to any one of claims 1 to 9.

Images