Cabinet-level multi-thread-oriented memory allocation method and system and server-free data center architecture

Abstract

The invention discloses a cabinet-level multi-thread-oriented memory allocation method and system and a server-free data center architecture, in the invention, all CPUs (Central Processing Unit) in the same cabinet are interconnected through a bus, a pooling memory is divided into an in-cabinet pooling memory and an out-cabinet pooling memory, and a CPU pool and the in-cabinet pooling memory in the same cabinet are interconnected; creating different NUMA nodes for the local memory, the in-cabinet pooling memory and the out-cabinet pooling memory, and identifying memory resource positions through NUMA node numbers; when memories are allocated to kernel data and user state data of a multi-thread process, physical memories are restrained to be allocated to pooling memories in a cabinet based on NUMA node numbers, so that cache consistency is guaranteed. According to the method and the device, the thread-intensive application can utilize the computing resources and the storage resources of the whole cabinet to restrain the cache consistency flow in the cabinet, so that the multi-thread process runs across the CPU pool in the cabinet, and the execution efficiency of the multi-thread process is effectively maintained.

Description

Technical Field

[0001] This invention relates to the field of computer technology, specifically to a rack-level multithreaded memory allocation method, system, and serverless data center architecture. Background Technology

[0002] Modern data centers are evolving towards decoupling storage and computing resources to improve resource utilization, scalability, and fault tolerance. Serverless data centers based on IPUs (I / O Processing Units) are a typical example of this architecture. For instance, the data center architecture disclosed in patent document CN117389938A uses the IPU as its core to build heterogeneous storage and computing resource pools, including CPUs (Central Processing Units), GPUs (Graphics Processing Units), FPGAs (Field Programmable Gate Arrays), RAM (Random Access Memory), SSDs (Solid State Disks), and HDDs (Hard Disk Drives). These pools are connected via a network, enabling interconnectivity and exhibiting high resource utilization, high scalability, and high fault tolerance.

[0003] In this type of data center architecture, memory resources mainly include the memory resources within the CPU pool and the memory resources across all memory pools. For example, the memory resource access method disclosed in patent document CN117707993A divides the memory resources within the CPU pool into local memory and L4 cache (Level 4 cache). Local memory is used only by the computing tasks running within this CPU pool and is used to store read-only kernel code and various kernel table entries, such as interrupt vector tables and process state record tables. The L4 cache is shared by all computing units within this CPU pool. The memory resources in all memory pools constitute pooled memory, uniformly addressed, and available to all computing tasks.

[0004] Multithreaded concurrency relies on shared memory, guaranteed by memory consistency and cache consistency mechanisms, and requires inter-core interrupt mechanisms. These mechanisms not only demand extremely low communication latency but also consume significant communication bandwidth. Therefore, traditional multithreaded execution is limited to the same cache consistency domain, typically confined to a single server node, and cannot run across nodes. In serverless data centers, high-latency network communication between memory and computing resource pools will compromise the efficiency of cache consistency mechanisms, causing multithreaded execution to fail correctly or efficiently in cross-CPU pool scenarios, thus failing to meet the needs of thread-intensive applications. Summary of the Invention

[0005] The purpose of this invention is to provide a rack-level multithreaded memory allocation method, system, and serverless data center architecture, enabling multithreaded processes to run across CPU pools within a serverless data center rack to meet the needs of thread-intensive applications.

[0006] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a rack-level multi-threaded memory allocation method, applied to multiple CPU pools in a serverless data center. The serverless data center includes multiple CPU pools, pooled memory interconnected with the multiple CPU pools via a network, and a control center for managing the multiple CPU pools and the pooled memory. Each CPU pool has local memory. All CPUs within all CPU pools in the same rack used for multi-threaded operation are interconnected via a bus. The pooled memory is divided into in-rack pooled memory and out-of-rack pooled memory based on the relative physical location of the memory resources and the rack. The in-rack pooled memory within the same rack is interconnected with all CPU pools used for multi-threaded operation. The method includes:

[0008] Create different NUMA (Non-Uniform Memory Access) nodes for local memory, in-cabinet pooled memory, and out-of-cabinet pooled memory, and identify the memory resource location by NUMA node number;

[0009] When allocating memory for kernel data and user-space data of multi-threaded processes, physical memory is constrained to be allocated to the pooled memory within the cabinet based on the NUMA node number. The pooled memory within the cabinet is used to ensure cache consistency of kernel data and user-space data of multi-threaded processes.

[0010] Furthermore, different NUMA nodes are created for local memory, in-facility pooled memory, and out-of-facility pooled memory, and the memory resource locations are identified by NUMA node numbers, including:

[0011] Create a NUMA node for local memory, the NUMA node carrying a NUMA node number used to identify the location of the NUMA node in the memory resource;

[0012] Send a memory request message to the control center, the memory request message including memory resource location preference information;

[0013] Receive an allocation response message from the control center, the allocation response message including relative memory resource location indication information;

[0014] Based on the allocation response message, create a NUMA node for the allocated memory resources located in the in-cabinet pooled memory or the out-of-cabinet pooled memory.

[0015] Furthermore, the step of identifying the memory resource location using NUMA node numbers includes:

[0016] Allocate NUMA node numbers within the first preset value range for local memory;

[0017] Assign NUMA node numbers within the second preset value range to the pooled memory within the cabinet;

[0018] Allocate NUMA node numbers within the third preset value range for off-cabinet pooled memory.

[0019] Furthermore, when allocating memory for kernel data, based on the NUMA node number, the allocation of physical memory within the cabinet's pooled memory is constrained, including:

[0020] When the system kernel requests kernel space memory, for different memory request functions, memory is allocated from NUMA nodes identified as in-cabinet pooled memory by modifying the allocation logic.

[0021] Furthermore, the method of allocating memory from NUMA nodes identified as pooled memory within the cabinet by modifying the allocation logic for different memory request functions includes:

[0022] For memory requests initiated by functions that require contiguous physical memory addresses, memory is allocated from the cache pool of the NUMA node identified as the pooled memory within the cabinet when the requested memory resources are not greater than a preset threshold; memory is allocated from the direct mapping area of ​​the NUMA node identified as the pooled memory within the cabinet when the requested memory resources are greater than the preset threshold.

[0023] For virtual memory requests initiated by functions that do not require contiguous physical memory addresses, a virtual memory policy is set when generating the virtual memory region to limit the NUMA nodes that can allocate physical memory to one or more NUMA nodes numbered as in-cabinet pooled memory.

[0024] Furthermore, when allocating memory for user-space data of multi-threaded processes, based on the NUMA node number, the physical memory allocation is constrained to the pooled memory within the cabinet, including:

[0025] Determine whether the process that initiated the user space memory request is a multi-threaded process;

[0026] For user space memory requests initiated by multi-threaded processes, a virtual memory policy is set when generating the virtual memory region to limit the NUMA nodes that can allocate physical memory to one or more NUMA nodes numbered as in-cabinet pooled memory.

[0027] For user space memory requests initiated by a single-threaded process, the range of NUMA nodes is not limited when generating the virtual memory region.

[0028] Secondly, the present invention provides a rack-level multi-threaded memory allocation system applied to multiple CPU pools in a serverless data center. The serverless data center includes the multiple CPU pools, pooled memory interconnected with the multiple CPU pools via a network, and a control center for managing the multiple CPU pools and the pooled memory. Each CPU pool has local memory. All CPUs within all CPU pools in the same rack for multi-threaded operation are interconnected via a bus. The pooled memory is divided into in-rack pooled memory and out-of-rack pooled memory based on the relative physical location of the memory resources and the rack. The in-rack pooled memory within the same rack is interconnected with all CPU pools for multi-threaded operation. The system includes:

[0029] The memory identification module is used to create different NUMA nodes for local memory, in-cabinet pooled memory, and out-of-cabinet pooled memory, and to identify the location of memory resources by NUMA node number;

[0030] The memory allocation module is used to allocate physical memory to the in-cabinet pooled memory based on the NUMA node number when allocating memory for kernel data and user-space data of multi-threaded processes. The in-cabinet pooled memory is used to ensure cache consistency of kernel data and user-space data of multi-threaded processes.

[0031] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the rack-level multithreaded memory allocation method described in the first aspect.

[0032] Fourthly, the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the rack-level multithreaded memory allocation method described in the first aspect.

[0033] Fifthly, the present invention provides a serverless data center architecture, including multiple CPU pools, pooled memory interconnected with the multiple CPU pools via a network, and a control center for managing the multiple CPU pools and the pooled memory. Each CPU pool has local memory. All CPUs within all CPU pools in the same rack for multi-threaded operation are interconnected via a bus. The pooled memory is divided into in-rack pooled memory and out-of-rack pooled memory based on the relative physical location of memory resources and the rack. The in-rack pooled memory within the same rack is interconnected with all CPU pools for multi-threaded operation. An operating system runs on each of the multiple CPU pools, and the operating system executes the steps of the rack-level multi-threaded memory allocation method described in the first aspect.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. In the rack-level multi-threaded memory allocation method provided by this invention, all CPUs in all CPU pools used for multi-threaded operation within the same rack are interconnected via a bus. Pooled memory is divided into rack-in pooled memory and rack-out pooled memory based on the relative physical location of memory resources and the rack. The rack-in pooled memory within the same rack is interconnected with all CPU pools used for multi-threaded operation, breaking the limitation that multi-threaded processes can only run in a single CPU pool. This enables thread-intensive applications to utilize the computing and storage resources of the entire rack, improving parallel processing capabilities.

[0036] 2. In the rack-level multi-threaded memory allocation method provided by this invention, different NUMA nodes are created for local memory, rack-pooled memory, and external rack-pooled memory, and the memory resource location is identified by the NUMA node number. When allocating memory for kernel data and user-space data of multi-threaded processes, physical memory is constrained to be allocated to the rack-pooled memory based on the NUMA node number. The rack-pooled memory is used to ensure cache consistency of kernel data and user-space data of multi-threaded processes. Through this memory layout and allocation strategy, the large amount of consistency traffic generated by the cache consistency protocol is restricted to the low-latency rack internal network, avoiding the transmission of cache consistency traffic between racks through the high-latency data center network, thereby greatly reducing network bandwidth consumption and communication latency. Furthermore, restricting the cache consistency communication domain to the low-latency rack also ensures that the cache consistency protocol can be completed quickly, effectively maintaining the execution efficiency of multi-threaded processes.

[0037] 3. The rack-level multithreaded memory allocation system, electronic equipment, program products, and serverless data center architecture provided by this invention have the same effect as the rack-level multithreaded memory allocation method provided. Attached Figure Description

[0038] Figure 1 This diagram illustrates the memory resources of an existing serverless data center architecture.

[0039] Figure 2 This is a flowchart of a rack-level multithreaded memory allocation method provided in an embodiment of the present invention.

[0040] Figure 3 This is a schematic diagram of memory resource layout as an example of an embodiment of the present invention.

[0041] Figure 4 This is a schematic diagram illustrating the location of a NUMA node, as exemplified in an embodiment of the present invention.

[0042] Figure 5 This is a flowchart illustrating the memory resource allocation process in an embodiment of the present invention.

[0043] Figure 6 This is a flowchart illustrating the kernel space memory allocation process in an embodiment of the present invention.

[0044] Figure 7 This is a flowchart illustrating the user memory allocation process in an embodiment of the present invention.

[0045] Figure 8 This is a schematic diagram of a rack-level multithreaded memory allocation system provided in an embodiment of the present invention.

[0046] Figure 9 This is a schematic diagram of an electronic device structure provided in an embodiment of the present invention.

[0047] Figure 10 This is a schematic diagram of a serverless data center architecture provided in an embodiment of the present invention. Detailed Implementation

[0048] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] To facilitate understanding and explanation of the technical solutions of the embodiments of the present invention, the relevant prior art will be described first.

[0050] Figure 1 This example illustrates the memory resources of an existing serverless data center architecture, such as... Figure 1As shown, the serverless data center architecture includes various CPU pools, memory pools, and a control center. The memory resources of the serverless data center architecture are divided into CPU pool memory resources and memory pool memory resources. CPU pool memory resources include local memory and L4 cache, while memory pool memory resources constitute pooled memory. Specifically: local memory can only be accessed by local computing tasks, meaning it can only be used by computing tasks running within its own CPU pool. It is primarily used to store read-only kernel code and various kernel tables, such as interrupt vector tables and process status record tables, and its size is fixed. The L4 cache is a cache allocated within the CPU pool's memory, shared by all computing units within the local CPU pool. A prefetch strategy can be used to pre-fetch data at hardware addresses in the pooled memory into the L4 cache. Pooled memory can be accessed through physical addresses; that is, the resource user accesses the memory resources stored in the pooled memory through the physical address to be accessed. Pooled memory is composed of memory resources from all memory pools and can be used by all computing tasks in the serverless data center architecture. It is primarily used to store user-related data, such as user code, user stacks, and memory-mapped files, and can be expanded as needed.

[0051] Existing serverless data center architectures do not consider multi-threaded operation across CPU pools in terms of memory resource management, and therefore cannot meet the needs of thread-intensive applications.

[0052] The following provides a further description of the rack-level multithreaded memory allocation method involved in the embodiments of the present invention.

[0053] This invention provides a rack-level multi-threaded memory allocation method applied to multiple CPU pools in a serverless data center. The serverless data center includes the multiple CPU pools, pooled memory interconnected with the multiple CPU pools via a network, and a control center for managing the multiple CPU pools and the pooled memory. Each CPU pool has local memory. All CPUs within all CPU pools in the same rack used for multi-threaded operation are interconnected via a bus. The pooled memory is divided into in-rack pooled memory and out-of-rack pooled memory based on the relative physical location of the memory resources and the rack. The in-rack pooled memory within the same rack is interconnected with all CPU pools used for multi-threaded operation. Figure 2 As shown, the rack-level multi-threaded memory allocation method runs on an operating system, which runs on the multiple CPU pools, and may include the following steps S11 to S12.

[0054] S11. Create different NUMA nodes for local memory, in-cabinet pooled memory, and out-of-cabinet pooled memory, and identify the memory resource location by NUMA node number.

[0055] S12. When allocating memory for kernel data and user-mode data of multi-threaded processes, physical memory is constrained to be allocated to the pooled memory within the cabinet based on the NUMA node number. The pooled memory within the cabinet is used to ensure cache consistency of kernel data and user-mode data of multi-threaded processes.

[0056] In this embodiment, all CPUs in all CPU pools used for multi-threaded operation within the same rack are interconnected via a bus. Pooled memory is divided into rack-side pooled memory and rack-side pooled memory based on the relative physical location of memory resources and the rack. The rack-side pooled memory within the same rack is interconnected with all CPU pools used for multi-threaded operation, breaking the limitation that multi-threaded processes can only run in a single CPU pool. This allows thread-intensive applications to utilize the computing and storage resources of the entire rack, improving parallel processing capabilities. In this embodiment, different NUMA nodes are created for local memory, in-rack pooled memory, and out-of-rack pooled memory, and the memory resource locations are identified by NUMA node numbers. When allocating memory for kernel data and user-space data of multi-threaded processes, physical memory is constrained to be allocated to the in-rack pooled memory based on the NUMA node numbers. The in-rack pooled memory is used to ensure cache consistency of kernel data and user-space data of multi-threaded processes. Through this memory layout and allocation strategy, a large amount of consistency traffic (such as cache invalidation and update messages) generated by cache consistency protocols, such as MESI (Modified Exclusive-Shared-Invalid), is restricted to the low-latency internal network of the rack, such as the CXL (Compute Express Link) interconnect network. This avoids cache consistency traffic being transmitted between racks through the high-latency data center network, thereby greatly reducing network bandwidth consumption and communication latency. Furthermore, restricting the cache consistency communication domain to the low-latency rack also ensures that the cache consistency protocol can complete quickly, effectively maintaining the execution efficiency of multi-threaded processes.

[0057] Figure 3This paper presents an example of a rack-level multithreaded memory resource layout. To meet the cache coherency and inter-core interrupt requirements of multithreaded processes running across CPU boxes within a rack, all CPUs within the rack are interconnected via a dedicated bus, such as the UPI bus for Intel CPUs and the CCIX bus for ARM. To ensure its own secure and stable operation, the operating system divides data in memory into kernel space and user space. Data in kernel space is shared by all processes running on the operating system; data in user space is unique to each process but shared by threads within that process. In-rack pooled memory enables cache coherency capabilities, such as interconnecting CPU and memory pools within the rack via the CXL3.X protocol, while off-rack pooled memory only supports memory access capabilities, such as interconnecting CPU and memory pools between racks via protocols supporting memory semantics, such as SUE (Scale-Up Ethernet) and ETH-X (Ethernet-based eXtensible). Appropriate data storage locations are selected based on data attributes, thus confining data requiring cache coherency within the rack, thereby reducing inter-rack communication overhead while meeting low-latency communication requirements. Specifically... Figure 3 As shown, the pooled memory of all racks is uniformly addressed. For example, memory resources addressed from 0 to x-1 are located in rack 1, and memory resources addressed from x to 2x-1 are located in rack 2. Racks 1 to N run system kernels 1 to N, respectively. The operating systems of racks 1 and 2 run multi-threaded processes 2 and 4. Threads 2.1 and 2.2 of process 2 can run in different CPU pools. The operating systems of racks 1 and N run single-threaded processes 1 and 3. By placing read-only kernel data in local memory near the CPU pool and placing kernel state data (such as kernel 1 state) in the rack-based pooled memory, kernel space sharing is achieved while maintaining performance, enabling all CPUs in the rack to be uniformly and efficiently managed by the same system kernel. Simultaneously, user-mode data of multi-threaded processes (such as processes 2 and 4) is placed in the rack-based pooled memory to meet the cache consistency requirements of multi-threaded processes.

[0058] In practical applications, the location of user-space data for single-threaded processes (such as processes 1 and 3) can be left unconstrained, as they do not require cache consistency. This maintains the flexibility and resource utilization of the architecture. No strict constraints are placed on memory allocation for single-threaded processes, allowing full utilization of all memory resources. This satisfies the performance requirements of multi-threading without compromising the inherent high resource utilization and scalability advantages of serverless data centers.

[0059] In one possible implementation, step S11 creates different NUMA nodes for local memory, in-cabinet pooled memory, and out-of-cabinet pooled memory, and identifies the memory resource location by NUMA node number. This may include the following steps S111 to S112.

[0060] S111. Create a NUMA node for local memory, wherein the NUMA node carries a NUMA node number used to identify the location of the NUMA node in the memory resource.

[0061] S112. Create different NUMA nodes for in-cabinet pooled memory and out-of-cabinet pooled memory, and identify the memory resource location by NUMA node number. Specifically, this may include:

[0062] S1121. Send a memory request message to the control center. The memory request message includes memory resource location preference information.

[0063] S1122. Receive the allocation response message from the control center, the allocation response message including relative location indication information of memory resources.

[0064] S1123. Based on the allocation response message, create a NUMA node for the allocated memory resources located in the in-cabinet pooled memory or the out-of-cabinet pooled memory.

[0065] In practical applications, local memory can be automatically discovered and the corresponding NUMA nodes can be created when the operating system starts up, while in-cabinet pooled memory and out-of-cabinet pooled memory can be applied for, added and released as needed.

[0066] In one possible implementation, the memory resource location is identified by NUMA node numbers in step S11. This can be achieved by: allocating NUMA node numbers within a first preset value range to local memory; allocating NUMA node numbers within a second preset value range to in-cabinet pooled memory; and allocating NUMA node numbers within a third preset value range to out-of-cabinet pooled memory. By allocating NUMA node numbers within different preset value ranges to local memory, in-cabinet pooled memory, and out-of-cabinet pooled memory, the memory resource locations can be effectively distinguished.

[0067] Figure 4 An example of a NUMA node location identification method is provided. Figure 4As shown, NUMA nodes numbered [0, RACK_NODES) are located in local memory; NUMA nodes numbered [RACK_NODES, REMOTE_NODES) are located in pooled memory within the cabinet; and NUMA nodes numbered [REMOTE_NODES, MAX_NUMNODES) are located in pooled memory outside the cabinet. Here, RACK_NODES, REMOTE_NODES, and MAX_NUMNODES are positive integers and can be customized as needed.

[0068] Figure 5 An example of a memory resource allocation process is provided. Figure 5 As shown, when existing memory resources are insufficient, the system kernel sends a memory request message to the control center via the IPU to request memory resources. The memory request message includes information such as the requester's location (e.g., the CPU pool number running the system kernel), memory size, and memory resource location preference (e.g., LOCAL or REMOTE). Upon receiving the memory request message, the control center allocates memory resources based on the memory resource location preference, requester's location, and memory size, and replies to the requester with relevant information via an allocation response message. The allocation response message includes relative location information of the memory resource (which indicates whether the allocated memory resource belongs to in-facility pooled memory or out-of-facility pooled memory, such as the memory pool number), read / write permissions, etc. After receiving the allocation response message, the system kernel creates a NUMA node for the allocated memory resource located in in-facility pooled memory or out-of-facility pooled memory. Specifically, the system kernel may set a node number based on the relative location of the memory resource and then register it as a NUMA node. If the relative location of the memory resource is within the rack, the node number is set to [RACK_NODES, REMOTE_NODES); otherwise, the node number is set to [REMOTE_NODES, MAX_NUMNODES]. The specific identification of the relative location of the memory resource can be achieved by the control center displaying its relative position to the resource provider based on a maintained global resource location and topology table, or by the resource user determining it themselves using an agreed-upon numbering rule. This invention does not impose any limitations on this.

[0069] In one possible implementation, when allocating memory for kernel data and user-space data of multi-threaded processes in S12, the allocation of physical memory to the pooled memory within the cabinet is constrained based on the NUMA node number, which may include the following steps S121 to S122.

[0070] S121. When allocating memory for kernel data, based on the NUMA node number, constrain the allocation of physical memory to the pooled memory within the cabinet.

[0071] S122. When allocating memory for user-mode data of a multi-threaded process, physical memory is constrained to be allocated to the pooled memory within the cabinet based on the NUMA node number.

[0072] In step S121, when the system kernel requests kernel space memory, for different memory request functions, memory is allocated from NUMA nodes identified as in-cabinet pooled memory by modifying the allocation logic.

[0073] In one possible implementation, the allocation of memory from the NUMA node identified as the in-cabinet pooled memory by modifying the allocation logic for different memory request functions may include the following steps S1211 to S1212.

[0074] S1211. For memory requests initiated by functions that require contiguous physical memory addresses, if the requested memory resources are not greater than a preset threshold, memory is allocated from the cache pool of the NUMA node identified as the pooled memory within the cabinet; if the requested memory resources are greater than the preset threshold, memory is allocated from the direct mapping area of ​​the NUMA node identified as the pooled memory within the cabinet.

[0075] S1212. For virtual memory requests initiated by functions that do not require contiguous physical memory addresses, when generating virtual memory regions, set a virtual memory policy to limit the NUMA nodes that can allocate physical memory to one or more NUMA nodes numbered as in-cabinet pooled memory.

[0076] In step S122, when allocating memory for user-mode data of a multi-threaded process, the physical memory is constrained to be allocated to the pooled memory within the cabinet based on the NUMA node number, which may include the following steps S1221 to S1223.

[0077] S1221. Determine whether the process that initiated the user space memory request is a multi-threaded process.

[0078] S1222. For user space memory requests initiated by multi-threaded processes, when generating virtual memory regions, set virtual memory policies to limit the NUMA nodes that can allocate physical memory to one or more NUMA nodes numbered as in-cabinet pooled memory.

[0079] S1223. For user space memory requests initiated by a single-threaded process, the range of NUMA nodes is not limited when generating the virtual memory region.

[0080] Figure 6This example illustrates a kernel space memory allocation process applicable to the Linux operating system, running in kernel mode. In Linux, data in kernel space is allocated by the kernel and shared by all threads, primarily through the `kmalloc` and `vmalloc` functions. The former allocates smaller, contiguous physical memory, while the latter allocates larger memory spaces. For example... Figure 6 As shown, when the kernel requests memory resources using the kmalloc function, if the requested memory resources are not greater than a preset threshold (e.g., 8KB), memory is allocated only in the cache pool (e.g., Slab or Slub pool) of the NUMA nodes in the rack-pooled memory (i.e., NUMA nodes numbered [RACK_NODES, REMOTE_NODES)). If the requested memory resources are greater than the preset threshold (e.g., 8KB), memory is allocated only in the direct mapping area of ​​the NUMA nodes numbered [RACK_NODES, REMOTE_NODES). When the kernel requests memory resources using the vmalloc function, a NUMA node preference is added to the current vmalloc implementation logic to constrain memory allocation to occur on the NUMA nodes in the rack-pooled memory. This means that when generating the virtual memory region, a virtual memory policy is set to limit the NUMA nodes that can allocate physical memory to one or more NUMA nodes numbered within the [RACK_NODES, REMOTE_NODES) region. Through this method, kernel data can be constrained within the local rack.

[0081] Figure 7 This example demonstrates a user memory allocation process applicable to the Linux operating system. Figure 7 As shown, when a process requests user-space memory by calling the memory allocation interface in user mode (such as malloc), it enters kernel mode. If the process is multi-threaded, a corresponding VMA (Virtual Memory Area) is generated, and the NUMA node that can allocate physical memory is set to the NUMA node of the pooled memory within the cabinet through vma->vm_policy, that is, one or more NUMA nodes with numbers in the [RACK_NODES, REMOTE_NODES) region are selected. If the process is not multi-threaded, a corresponding VMA is generated, but the NUMA node for allocating physical memory is not restricted.

[0082] It should be noted that the above examples are exemplary illustrations of the relevant memory allocation functions for the Linux operating system. The memory allocation functions may differ for different types of operating systems, but the allocation strategy of this invention is equally applicable.

[0083] The following is a further description of the rack-level multithreaded memory allocation system according to embodiments of the present invention. The memory allocation system described below can be referred to in correspondence with the memory allocation method described above.

[0084] like Figure 8 As shown, this embodiment of the invention provides a rack-level multi-threaded memory allocation system applied to multiple CPU pools in a serverless data center. The serverless data center includes the multiple CPU pools, pooled memory interconnected with the multiple CPU pools via a network, and a control center for managing the multiple CPU pools and the pooled memory. Each CPU pool has local memory. All CPUs within all CPU pools in the same rack used for multi-threaded operation are interconnected via a bus. The pooled memory is divided into in-rack pooled memory and out-of-rack pooled memory based on the relative physical location of the memory resources and the rack. The in-rack pooled memory within the same rack is interconnected with all CPU pools used for multi-threaded operation. The system includes:

[0085] Memory identification module 11 is used to create different NUMA nodes for local memory, in-cabinet pooled memory and out-of-cabinet pooled memory, and to identify the location of memory resources by NUMA node number;

[0086] The memory allocation module 12 is used to allocate physical memory to the in-cabinet pooled memory based on the NUMA node number when allocating memory for kernel data and user-space data of multi-threaded processes. The in-cabinet pooled memory is used to ensure cache consistency of kernel data and user-space data of multi-threaded processes.

[0087] like Figure 9 As shown, this embodiment of the invention also provides an electronic device, including a memory 21, a processor 22, and a computer program stored on the memory 21 and executable on the processor 22. When the computer program is executed by the processor 22, it implements the steps of the rack-level multithreaded memory allocation method in any of the above embodiments.

[0088] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the rack-level multithreaded memory allocation method in any of the above embodiments. The program code for implementing the method of this invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the steps of the method of this invention to be implemented.

[0089] The memory allocation system, electronic device, and computer program product provided in this embodiment have the same beneficial effects as the memory allocation method described above, and will not be repeated here.

[0090] like Figure 10 As shown, this embodiment of the invention also provides a serverless data center architecture, including multiple CPU pools, pooled memory interconnected with the multiple CPU pools via a network, and a control center for managing the multiple CPU pools and the pooled memory. Each CPU pool has local memory. All CPUs in all CPU pools within the same rack for multi-threaded operation are interconnected via a bus. The pooled memory is divided into in-rack pooled memory and out-of-rack pooled memory based on the relative physical location of memory resources and the rack. The in-rack pooled memory within the same rack is interconnected with all CPU pools for multi-threaded operation. An operating system runs on each of the multiple CPU pools, and the operating system executes the steps of the rack-level multi-threaded memory allocation method in any of the above embodiments.

[0091] The serverless data center architecture provided in this embodiment is based on the implementation of a rack-level multi-threaded memory allocation method, which has the same beneficial effects as the memory allocation method described above, and will not be repeated here.

[0092] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0093] It should also be noted that relational terms such as "first" and "second" in this specification are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The term "and / or" includes any and all combinations of one or more of the associated listed items. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0094] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A rack-level multithreaded memory allocation method, applied to multiple CPU pools in a serverless data center, wherein the serverless data center includes the multiple CPU pools, pooled memory interconnected with the multiple CPU pools via a network, and a control center for managing the multiple CPU pools and the pooled memory, wherein each CPU pool has local memory; characterized in that: All CPUs within a single rack that are used for multi-threaded operation are interconnected via a bus. The pooled memory is divided into in-rack pooled memory and out-of-rack pooled memory based on the relative physical location of the memory resources and the rack. The in-rack pooled memory within the same rack is interconnected with all CPU pools used for multi-threaded operation. The method includes: Create different non-uniform memory access NUMA nodes for local memory, in-cabinet pooled memory, and out-of-cabinet pooled memory, and identify the memory resource location by NUMA node number; When allocating memory for kernel data and user-space data of multi-threaded processes, physical memory is constrained to be allocated to the pooled memory within the cabinet based on the NUMA node number. The pooled memory within the cabinet is used to ensure cache consistency of kernel data and user-space data of multi-threaded processes.

2. The memory allocation method for rack-level multithreading according to claim 1, characterized in that: Different NUMA nodes are created for local memory, in-facility pooled memory, and out-of-facility pooled memory, and the memory resource locations are identified by NUMA node numbers, including: Create a NUMA node for local memory, the NUMA node carrying a NUMA node number used to identify the location of the NUMA node in the memory resource; Send a memory request message to the control center, the memory request message including memory resource location preference information; Receive an allocation response message from the control center, the allocation response message including relative memory resource location indication information; Based on the allocation response message, create a NUMA node for the allocated memory resources located in the in-cabinet pooled memory or the out-of-cabinet pooled memory.

3. The memory allocation method for rack-level multithreading according to claim 1, characterized in that: The method of identifying memory resource locations using NUMA node numbers includes: Allocate NUMA node numbers within the first preset value range for local memory; Assign NUMA node numbers within the second preset value range to the pooled memory within the cabinet; Allocate NUMA node numbers within the third preset value range for off-cabinet pooled memory.

4. The memory allocation method for rack-level multithreading according to claim 1, characterized in that: When allocating memory for kernel data, based on the NUMA node number, the physical memory allocation is constrained to the pooled memory within the cabinet, including: When the system kernel requests kernel space memory, for different memory request functions, memory is allocated from NUMA nodes identified as in-cabinet pooled memory by modifying the allocation logic.

5. A memory allocation method for rack-level multithreading according to claim 4, characterized in that: The method of allocating memory from NUMA nodes identified as pooled memory within the cabinet by modifying the allocation logic for different memory request functions includes: For memory requests initiated by functions that require contiguous physical memory addresses, memory is allocated from the cache pool of the NUMA node identified as the pooled memory within the cabinet when the requested memory resources are not greater than a preset threshold; memory is allocated from the direct mapping area of ​​the NUMA node identified as the pooled memory within the cabinet when the requested memory resources are greater than the preset threshold. For virtual memory requests initiated by functions that do not require contiguous physical memory addresses, a virtual memory policy is set when generating the virtual memory region to limit the NUMA nodes that can allocate physical memory to one or more NUMA nodes numbered as in-cabinet pooled memory.

6. The memory allocation method for rack-level multithreading according to claim 1, characterized in that: When allocating memory for user-space data of multi-threaded processes, based on the NUMA node number, the physical memory allocation is constrained to the pooled memory within the cabinet, including: Determine whether the process that initiated the user space memory request is a multi-threaded process; For user space memory requests initiated by multi-threaded processes, a virtual memory policy is set when generating the virtual memory region to limit the NUMA nodes that can allocate physical memory to one or more NUMA nodes numbered as in-cabinet pooled memory. For user space memory requests initiated by a single-threaded process, the range of NUMA nodes is not limited when generating the virtual memory region.

7. A rack-level multi-threaded memory allocation system, applied to multiple CPU pools in a serverless data center, the serverless data center including the multiple CPU pools, pooled memory interconnected with the multiple CPU pools via a network, and a control center for managing the multiple CPU pools and the pooled memory, wherein each CPU pool has local memory; characterized in that: All CPUs within a single rack that are used for multi-threaded operation are interconnected via a bus. The pooled memory is divided into in-rack pooled memory and out-of-rack pooled memory based on the relative physical location of the memory resources and the rack. The in-rack pooled memory within the same rack is interconnected with all CPU pools used for multi-threaded operation. The system includes: The memory identification module is used to create different non-uniform memory access NUMA nodes for local memory, in-cabinet pooled memory and out-of-cabinet pooled memory, and to identify the location of memory resources by NUMA node number; The memory allocation module is used to allocate physical memory to the in-cabinet pooled memory based on the NUMA node number when allocating memory for kernel data and user-space data of multi-threaded processes. The in-cabinet pooled memory is used to ensure cache consistency of kernel data and user-space data of multi-threaded processes.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by the processor, it implements the steps of the rack-level multithreaded memory allocation method according to any one of claims 1-6.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the rack-level multithreaded memory allocation method according to any one of claims 1-6.

10. A serverless data center architecture, comprising multiple CPU pools, pooled memory interconnected with the multiple CPU pools via a network, and a control center for managing the multiple CPU pools and the pooled memory, wherein each CPU pool has local memory; characterized in that: All CPUs in all CPU pools used for multi-threaded operation within the same rack are interconnected via a bus. The pooled memory is divided into in-rack pooled memory and out-of-rack pooled memory based on the relative physical location of memory resources and the rack. The in-rack pooled memory within the same rack is interconnected with all CPU pools used for multi-threaded operation. An operating system runs on the multiple CPU pools, and the operating system executes the steps of the rack-level multi-threaded memory allocation method according to any one of claims 1-6.

Images