A memory resource arrangement method and device, electronic equipment and storage medium

Abstract

The embodiment of the application discloses a memory resource arrangement method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: determining a data memory placement combination based on a data set to be placed in the memory resource and at least one memory region; obtaining a memory access cost corresponding to each data memory placement combination; constructing a cost function for representing a total memory access cost corresponding to a placement arrangement scheme based on the memory access cost corresponding to each data memory placement combination; performing solving processing on the cost function to obtain a target placement arrangement scheme, wherein the target placement arrangement scheme makes the function value of the cost function meet a predetermined requirement; and placing the data block in the memory region based on the obtained target placement arrangement scheme. The application is suitable for the arrangement of memory resources.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and storage medium for arranging memory resources. Background Technology

[0002] In modern computer systems, especially in cloud computing, big data processing, and high-performance computing, the efficiency of memory resource access is one of the key factors determining the overall system performance. With the rapid increase in the number of processor cores, system architectures have evolved from traditional symmetric multiprocessing architectures to more complex architectures such as Non-Uniform Memory Access (NUMA). Taking NUMA architecture as an example, the processor is divided into multiple nodes, each with its own local memory. The processor accesses its local memory much faster than it accesses the remote memory of other nodes. This non-uniform access characteristic makes the spatial planning and arrangement of memory resources a decisive factor in system performance.

[0003] Currently, mainstream automated planning and orchestration schemes are based on the operating system kernel's awareness and scheduling mechanisms. Taking the Linux operating system as an example, the AutoNUMA (Automatic Non-Consistent Memory Access) balancing mechanism was introduced starting with kernel version 3.8, and user-space daemons such as numad were subsequently developed. The basic working principle of this type of scheme is: the kernel periodically samples the memory page access of processes, and when it detects that the proportion of remote access to a certain memory page exceeds a preset threshold, the system asynchronously starts a migration thread in the background to migrate the page from the remote node to the local node, which may be accompanied by cross-node scheduling of CPU tasks.

[0004] However, the judgment basis of such schemes is limited to a few coarse-grained sampling indicators such as "number of remote accesses" and "node distance," resulting in a large number of invalid or inefficient migration behaviors, which in turn leads to serious cache invalidation overhead. For example, these migrations may trigger a global flush of the TLB (Translation Lookaside Buffer), multi-level cache (L1 / L2 / L3) invalidation, and page fault handling. This cache invalidation overhead can cause performance jitter in latency-sensitive applications such as databases and real-time systems, and in severe cases, may lead to service timeouts. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method, apparatus, electronic device, and storage medium for arranging memory resources, which effectively improves the access efficiency of memory resources.

[0006] In a first aspect, embodiments of the present invention provide a method for orchestrating memory resources, the memory resources including at least one memory region, the method comprising: determining data memory placement combinations based on a data set to be placed into the memory resource and the at least one memory region, wherein the data set includes at least one data block, and the data memory placement combination includes the data block and the memory region where the data block is to be placed; obtaining the memory access cost corresponding to each data memory placement combination; constructing a cost function characterizing the total memory access cost corresponding to a placement orchestration scheme based on the memory access cost corresponding to each data memory placement combination, wherein the total memory access cost corresponding to the placement orchestration scheme is the sum of the memory access costs corresponding to each data memory placement combination in the placement orchestration scheme; solving the cost function to obtain a target placement orchestration scheme, the target placement orchestration scheme causing the function value of the cost function to meet predetermined requirements; and placing the data block into the memory region based on the obtained target placement orchestration scheme.

[0007] In one embodiment, obtaining the memory access cost corresponding to each of the data memory placement combinations includes: obtaining the memory access cost corresponding to each of the data memory placement combinations based on the memory access feature information and preset weight information corresponding to the memory region.

[0008] In one embodiment, the memory access characteristic information corresponding to the memory region includes at least one of the following: access frequency, access distance, task weight, scheduling overhead, cache hit rate, bandwidth contention, access mode, prefetch penalty, power consumption coefficient, fragmentation coefficient, interconnect congestion, and hot throttling.

[0009] In one implementation, the process of solving the cost function includes: solving the cost function based on predefined constraints so that the target placement and arrangement scheme satisfies the constraints.

[0010] In one implementation, the constraints include at least one of the following: unique assignment constraint, capacity constraint, and allowable domain constraint; wherein the unique assignment constraint stipulates that each data block can only be placed in one memory region, the capacity constraint stipulates that the space occupied by the data block placed in each memory region cannot exceed the memory capacity of the memory region, and the allowable domain constraint stipulates the memory regions where the data block is allowed to be placed.

[0011] In one implementation, the cost function is constructed as the sum of the products of the memory access costs corresponding to each of the data memory placement combinations and a predetermined decision variable, wherein the decision variable is used to decide whether the memory access costs corresponding to the data memory placement combinations are included in the total memory access costs; the process of solving the cost function based on predefined constraints includes: solving the cost function based on predefined constraints with the objective of minimizing the function value to obtain the values ​​of the decision variables; and obtaining a target placement orchestration scheme based on the obtained decision variable values.

[0012] In one implementation, the step of solving the cost function based on predefined constraints to obtain the decision variable values ​​includes: generating an initial feasible solution for the cost function using a predetermined heuristic construction algorithm; performing iterative search processing in the solution space of the cost function using a metaheuristic global search algorithm based on the initial feasible solution to obtain optimized candidate solutions where the function value of the cost function is lower than the function value of the cost function corresponding to the initial feasible solution; and obtaining the decision variable values ​​based on the optimized candidate solutions.

[0013] In one implementation, obtaining the decision variable value based on the candidate optimization solution includes: pruning the branch using a predetermined branch-and-bound algorithm with the function value of the cost function corresponding to the candidate optimization solution as a global upper bound, to obtain a target optimal solution where the function value of the cost function is lower than the function value of the cost function corresponding to the candidate optimization solution; and obtaining the decision variable value based on the target optimal solution.

[0014] In one implementation, generating an initial feasible solution for the cost function using a predetermined heuristic construction algorithm includes: sorting all the data groups in descending order of memory usage; sequentially allocating the allowable memory region with the smallest remaining capacity after placing the data group in each of the descendingly sorted data groups, and updating the available capacity of the selected memory region; and obtaining the initial feasible solution based on the allocation scheme formed after all the data groups have been placed.

[0015] In one implementation, generating an initial feasible solution to the cost function using a predetermined heuristic construction algorithm includes: identifying at least two greedy algorithm variants, wherein the sorting rules or selection criteria of the at least two greedy algorithm variants differ; executing each of the greedy algorithm variants in parallel, sorting all the data groups according to the sorting rules specified by the variant; sequentially allocating memory regions for each of the sorted data groups according to the selection criteria specified by the variant, and updating the available capacity of the selected memory regions; and collecting the allocation schemes obtained from the parallel execution of all the greedy algorithm variants to obtain multiple initial feasible solutions.

[0016] In one implementation, the sorting rules include at least one of the following: descending order of memory usage, ascending order of memory usage, descending order of access frequency, and descending order of task weight; and / or the selection criteria include at least one of the following: selecting the option with the smallest remaining capacity after placement, selecting the option with the largest remaining capacity after placement, selecting the option with the smallest access latency, and selecting the option with the largest currently available capacity.

[0017] In one embodiment, the metaheuristic global search algorithm includes a parallel simulated annealing algorithm. The step of performing search and iteration processing in the solution space of the cost function based on the initial feasible solution includes: taking the initial feasible solution as a starting point, exploring multiple neighborhoods or temperature states in parallel, and obtaining optimized candidate solutions where the function value of the cost function is lower than the function value of the cost function corresponding to the initial feasible solution.

[0018] In one implementation, after placing the data blocks into the memory region based on the acquired target placement orchestration scheme, the method further includes: real-time monitoring of memory access performance indicators for each data group in the memory region, the memory access performance indicators including at least one of the following: average memory access latency, remote access ratio, memory bandwidth utilization, and cache hit rate; if the memory access performance indicators do not meet preset conditions, selecting key parameters from the memory access quantification information of the memory region where the data groups are placed, and adjusting the weight values ​​corresponding to each key parameter; based on the key parameters and the adjusted weight values, re-acquiring the memory access cost of the data memory placement combination; then constructing and solving the cost function using the re-acquiring memory access cost to obtain an updated target placement orchestration scheme; and placing the data blocks into the updated memory region based on the updated target placement orchestration scheme until the memory access performance indicators meet preset conditions.

[0019] In one embodiment, the memory resources include at least one of the following: local memory of a non-consistent memory access node, dynamic random access memory, high-bandwidth memory, persistent memory, at least two levels of storage in a hierarchical storage system, video memory of a graphics processor, extended memory devices or memory pools connected via a compute fast link protocol.

[0020] Secondly, embodiments of the present invention also provide a memory resource orchestration apparatus, wherein the memory resource includes at least one memory region, the apparatus comprising: a determining unit, configured to determine a data memory placement combination based on a data set to be placed into the memory resource and the at least one memory region, wherein the data set includes at least one data block, and the data memory placement combination includes the data block and a memory region where the data block is to be placed; an obtaining unit, configured to obtain the memory access cost corresponding to each data memory placement combination; a constructing unit, configured to construct a cost function characterizing the total memory access cost corresponding to a placement orchestration scheme based on the memory access cost corresponding to each data memory placement combination, wherein the total memory access cost corresponding to the placement orchestration scheme is the sum of the memory access costs corresponding to each data memory placement combination in the placement orchestration scheme; a solving unit, configured to solve the cost function to obtain a target placement orchestration scheme, wherein the target placement orchestration scheme makes the function value of the cost function meet predetermined requirements; and a placing unit, configured to place the data block into the memory region based on the obtained target placement orchestration scheme.

[0021] In one embodiment, the acquisition unit is specifically used to: acquire the memory access cost corresponding to each data memory placement combination based on the memory access feature information and preset weight information corresponding to the memory region.

[0022] In one embodiment, the memory access characteristic information corresponding to the memory region includes at least one of the following: access frequency, access distance, task weight, scheduling overhead, cache hit rate, bandwidth contention, access mode, prefetch penalty, power consumption coefficient, fragmentation coefficient, interconnect congestion, and hot throttling.

[0023] In one embodiment, the solving unit includes a solving module, configured to solve the cost function based on predefined constraints, so that the target placement arrangement scheme satisfies the constraints.

[0024] In one implementation, the constraints include at least one of the following: unique assignment constraint, capacity constraint, and allowable domain constraint; wherein the unique assignment constraint stipulates that each data block can only be placed in one memory region, the capacity constraint stipulates that the space occupied by the data block placed in each memory region cannot exceed the memory capacity of the memory region, and the allowable domain constraint stipulates the memory regions where the data block is allowed to be placed.

[0025] In one implementation, the cost function is constructed as the sum of the products of the memory access cost corresponding to each of the data memory placement combinations and a predetermined decision variable, wherein the decision variable is used to decide whether the memory access cost corresponding to the data memory placement combination is included in the total memory access cost; the solution module is specifically used to: solve the cost function based on predefined constraints, with the goal of minimizing the function value, to obtain the value of the decision variable; and obtain the target placement orchestration scheme based on the obtained decision variable value.

[0026] In one implementation, the solution module is specifically used to: generate an initial feasible solution for the cost function using a predetermined heuristic construction algorithm; perform iterative search processing in the solution space of the cost function based on the initial feasible solution using a metaheuristic global search algorithm to obtain optimized candidate solutions where the function value of the cost function is lower than the function value of the cost function corresponding to the initial feasible solution; and obtain the decision variable value based on the optimized candidate solution.

[0027] In one implementation, the solution module is specifically used to: prune the cost function corresponding to the candidate optimization solution using a predetermined branch and bound algorithm, taking the function value of the cost function as the global upper bound, to obtain a target optimal solution where the function value of the cost function is lower than the function value of the cost function corresponding to the candidate optimization solution; and obtain the decision variable value based on the target optimal solution.

[0028] In one implementation, the solution module is specifically used to: sort all the data groups in descending order of memory usage; sequentially select the allowable memory region with the smallest remaining capacity after placing the data group in the descending order of the data groups, and update the available capacity of the selected memory region; and obtain the initial feasible solution based on the allocation scheme formed after placing all the data groups.

[0029] In one implementation, the solution module is specifically configured to: determine at least two greedy algorithm variants, wherein the sorting rules or selection criteria of the at least two greedy algorithm variants differ; execute each of the greedy algorithm variants in parallel, and sort all the data groups according to the sorting rules specified by the variants; sequentially select a memory region for allocation for each of the sorted data groups according to the selection criteria specified by the variants, and update the available capacity of the selected memory region; collect the allocation schemes obtained by executing all the greedy algorithm variants in parallel, and obtain multiple initial feasible solutions.

[0030] In one implementation, the sorting rules include at least one of the following: descending order of memory usage, ascending order of memory usage, descending order of access frequency, and descending order of task weight; and / or the selection criteria include at least one of the following: selecting the option with the smallest remaining capacity after placement, selecting the option with the largest remaining capacity after placement, selecting the option with the smallest access latency, and selecting the option with the largest currently available capacity.

[0031] In one implementation, the metaheuristic global search algorithm includes a parallel simulated annealing algorithm, and the solution module is specifically used to: take the initial feasible solution as a starting point, explore multiple neighborhoods or temperature states in parallel, and obtain optimized candidate solutions where the function value of the cost function is lower than the function value of the cost function corresponding to the initial feasible solution.

[0032] In one embodiment, the apparatus further includes: a monitoring unit, configured to monitor in real time the memory access performance indicators of each data group in the memory region after the data block is placed in the memory region according to the acquired target placement orchestration scheme, the memory access performance indicators including at least one of the following: average memory access latency, remote access ratio, memory bandwidth utilization, and cache hit rate; and an adjustment unit, configured to, when the memory access performance indicators do not meet preset conditions, select key parameters from the memory access quantification information of the memory region where the data group is placed, adjust the weight values ​​corresponding to each key parameter, re-acquire the memory access cost of the data memory placement combination based on the key parameters and the adjusted weight values, construct and solve the cost function through the re-acquired memory access cost to obtain an updated target placement orchestration scheme, and place the data block in the updated memory region according to the updated target placement orchestration scheme until the memory access performance indicators meet preset conditions.

[0033] In one embodiment, the memory resources include at least one of the following: local memory of a non-consistent memory access node, dynamic random access memory, high-bandwidth memory, persistent memory, at least two levels of storage in a hierarchical storage system, video memory of a graphics processor, extended memory devices or memory pools connected via a compute fast link protocol.

[0034] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising: a processor and a memory, the processor being electrically connected to the memory; the memory being used to store executable program code; the processor running a program corresponding to the executable program code by reading the executable program code stored in the memory, thereby implementing the memory resource arrangement method provided in any one of the embodiments of the present invention.

[0035] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement any of the memory resource arrangement methods provided in the embodiments of the present invention.

[0036] The memory resource orchestration method, apparatus, electronic device, and storage medium provided by embodiments of the present invention can determine data memory placement combinations based on a data set to be placed in a memory region and a memory region, obtain the memory access cost corresponding to each data memory placement combination, construct a cost function to characterize the total memory access cost of the placement orchestration scheme based on each memory access cost, solve the cost function to obtain a target placement orchestration scheme that makes the cost function value meet predetermined requirements, and place data blocks into the memory region accordingly based on the target placement orchestration scheme. In this way, by constructing a cost function to characterize the total memory access cost corresponding to the placement orchestration scheme based on the memory access cost corresponding to each data memory placement combination, the memory orchestration problem is formalized into a solvable mathematical model, and a global optimization solution is performed based on the quantized memory access cost. Therefore, it can systematically reduce cross-node remote access latency and bandwidth contention, reduce unnecessary page migration and cache invalidation overhead, and thus effectively improve the access efficiency of memory resources. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 A flowchart illustrating a method for arranging memory resources according to an embodiment of the present invention; Figure 2 A flowchart illustrating a hybrid solution method for arranging memory resources provided in an embodiment of the present invention; Figure 3 A flowchart illustrating a greedy algorithm for obtaining an initial feasible solution in a memory resource arrangement method provided in an embodiment of the present invention; Figure 4 A detailed flowchart of a memory resource orchestration method provided for embodiments of the present invention; Figure 5 A schematic diagram of a memory resource orchestration apparatus provided for an embodiment of the present invention; Figure 6 A schematic diagram of an electronic device provided as an embodiment of the present invention. Detailed Implementation

[0039] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0040] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0041] In a first aspect, embodiments of the present invention provide a method for arranging memory resources, which effectively improves the access efficiency of memory resources.

[0042] like Figure 1 As shown, the embodiment of the present invention provides a method for arranging memory resources, wherein the memory resources include at least one memory region, and the method may include: S11, based on the data set to be placed into the memory resource and the at least one memory region, determine a data memory placement combination, wherein the data set includes at least one data block, and the data memory placement combination includes the data block and the memory region to which the data block is to be placed; The orchestration mentioned in this embodiment of the invention refers to how to place data blocks in the data set into memory resources. Specifically, it is necessary to plan the placement location of each data block, that is, which memory area each data block is placed in.

[0043] In this step, we first obtain the set of data to be orchestrated (i.e., placed into memory resources) and the available memory resources in the system. The memory resources can contain one or more physically or logically partitioned memory regions, such as the nodes in a NUMA architecture. The data set consists of several data blocks, each corresponding to a group of memory objects with strong locality of access characteristics (such as processes, containers, or cooperative task groups). Then, based on the data set to be orchestrated and the available memory regions, we determine the data memory placement combinations, i.e., data block and memory region pairs, to indicate that the data block can be placed in that memory region. Specifically, we can obtain possible pairings of data blocks and memory regions through enumeration or some algorithm to form data memory placement combinations. For example, we can traverse the Cartesian product of the set of memory objects (data sets) to be placed and the set of candidate memory regions to generate all possible (data set, memory region) pairs.

[0044] S12, obtain the memory access cost corresponding to each of the data memory placement combinations; This step calculates the memory access cost for each data memory placement combination generated in step S11. The memory access cost quantifies the overall overhead expected by one or more processor cores in the system to access a data block after it has been placed in a specific memory region. This memory access cost can be obtained in various ways, such as through sampling statistics based on historical access patterns, real-time monitoring based on hardware performance counters, or inference based on machine learning prediction models. This overall overhead can incorporate multiple factors, such as access frequency, physical distance, task priority, cache hit rate, bandwidth contention, and access patterns. Specifically, the memory access cost of placing a data block in a memory region can be calculated based on the memory access characteristics of that region, such as access distance and bandwidth contention coefficient.

[0045] S13, based on the memory access cost corresponding to each data memory placement combination, construct a cost function to characterize the total memory access cost corresponding to the placement orchestration scheme, wherein the total memory access cost corresponding to the placement orchestration scheme is the sum of the memory access costs corresponding to each data memory placement combination in the placement orchestration scheme; In this embodiment of the invention, the memory orchestration problem is formalized into a mathematical optimization model, which includes a cost function.

[0046] It is understandable that the placement orchestration scheme specifies the memory regions where each data group is placed, i.e., it selects several data memory placement combinations. Therefore, each placement orchestration scheme obviously corresponds to a total memory access cost, which can be defined as the sum of the memory access costs corresponding to all selected data memory placement combinations in the scheme. Different placement orchestration schemes may have different total memory access costs. Therefore, the purpose of this step is to construct a cost function characterizing the total memory access cost of a placement orchestration scheme, and then solve this cost function to obtain a placement orchestration scheme, i.e., a target placement orchestration scheme. This target placement orchestration scheme makes the function value of the cost function, i.e., the total memory access cost, meet a predetermined requirement, such as minimizing the total memory access cost, thereby transforming the memory orchestration problem into a solvable mathematical problem.

[0047] The cost function can be constructed as a summation expression of the memory access costs corresponding to each data memory placement combination determined in step S11. Specifically, it is constructed as the sum of the products of the memory access costs corresponding to each data memory placement combination and the decision variables. The independent variable is the decision variable, and the dependent variable, i.e., the function value, is the total memory access cost. The decision variables are used to decide the selected data memory placement combination in the placement orchestration scheme, that is, to decide whether the memory access cost corresponding to each data memory placement combination is included in the total memory access cost. If the combination is selected, the memory access cost of the combination is included in the total memory access cost. Different combinations result in different total memory access costs. Therefore, according to the predetermined requirements for the function value (i.e., the total memory access cost), such as minimizing the function value, the cost function is solved, i.e., the decision variables are solved, thereby obtaining the selected data memory placement combination and the target placement orchestration scheme.

[0048] In addition, mathematical optimization models can be fitted with constraints to ensure that the solution is truly feasible.

[0049] S14, Solve the cost function to obtain a target placement arrangement scheme, wherein the target placement arrangement scheme makes the function value of the cost function meet predetermined requirements; In this step, the cost function is solved according to predetermined requirements for the function value (i.e., total memory access cost), thereby obtaining a target placement orchestration scheme that meets the predetermined requirements. In specific implementations, any one or more optimization strategies can be used to solve the cost function. For example, the solution process may include deterministic algorithms, heuristic algorithms, metaheuristic algorithms, machine learning methods, or combinations thereof.

[0050] The predetermined requirements may include: making the cost function value approach its minimum, less than or equal to a preset threshold, showing improvement compared to the initial solution, or achieving global optimum or near-optimal under acceptable computational resource and time constraints. These predetermined requirements can be dynamically adjusted based on system load characteristics, real-time requirements, or user configuration, and are not limited to fixed convergence conditions or optimality criteria.

[0051] S15, based on the obtained target placement arrangement scheme, the data block is placed into the memory area accordingly.

[0052] This step executes the actual memory resource orchestration operation based on the solved target placement orchestration scheme. Specifically, each data block in the dataset is placed or migrated according to the target memory region indicated by the scheme. The execution method can employ one or more memory placement techniques. For example, it can use the NUMA memory binding interface provided by the operating system to set the policy, modify the memory allocator's allocation logic to ensure newly allocated pages directly fall into the specified region, trigger cross-region migration of existing data pages, or combine hardware mechanisms to implement address remapping. This step transforms the optimized target placement orchestration scheme into the actual physical memory layout, ensuring that the memory access behavior in the system is consistent with the expectations of the cost model, thereby reducing the overall memory access cost and meeting the predetermined performance requirements.

[0053] The memory resource orchestration method provided by the embodiments of the present invention can determine data memory placement combinations based on the data set to be placed in the memory region and the memory region, obtain the memory access cost corresponding to each data memory placement combination, construct a cost function to characterize the total memory access cost of the placement orchestration scheme based on each memory access cost, solve the cost function to obtain a target placement orchestration scheme that makes the cost function value meet predetermined requirements, and place data blocks into the memory region accordingly based on the target placement orchestration scheme. In this way, by constructing a cost function to characterize the total memory access cost corresponding to the placement orchestration scheme based on the memory access cost corresponding to each data memory placement combination, the memory orchestration problem is formalized into a solvable mathematical model, and a global optimization solution is performed based on the quantized memory access cost. Therefore, it can systematically reduce cross-node remote access latency and bandwidth contention, reduce unnecessary page migration and cache invalidation overhead, and thus effectively improve the access efficiency of memory resources.

[0054] Specifically, the memory resources applicable to the embodiments of the present invention may include: local memory of non-consistent memory access nodes, dynamic random access memory, high bandwidth memory, persistent memory, at least two levels of storage in hierarchical storage, video memory of graphics processors, extended memory devices or memory pools connected by computing fast link protocols, etc.

[0055] For example, the memory resources may include any one or more of the following types of storage media or devices: Local memory of non-uniform memory access nodes: Local dynamic random access memory (DRAM) mounted on a standard NUMA node. Dynamic Random Access Memory (DRAM): The standard type of general-purpose memory; High Bandwidth Memory (HBM): A type of stacked memory that is tightly integrated with a processor or accelerator and provides extremely high bandwidth. It is often used in high-performance computing and AI chips. Persistent Memory (PMEM): Located on the memory bus but non-volatile, its access latency and characteristics differ from DRAM; At least two storage tiers in tiered storage: for example, a two-tier memory system consisting of DRAM and PMEM, or a two-tier system consisting of HBM and DRAM; Graphics Processor Memory: Refers to the dedicated video memory on the GPU (Graphics Processing Unit) board, used to store data for GPU computing tasks; Extended memory devices or memory pools connected via Compute Express Link (CXL) protocols: These refer to memory expansion cards or pooled memory resources connected via high-speed interconnect protocols such as CXL (Compute Express Link). They are semantically accessible to the host CPU but have different access latency and bandwidth characteristics.

[0056] For example, in a heterogeneous memory system integrating HBM and DRAM, latency-sensitive core data (such as hot data) can be placed in the high-bandwidth HBM, while cold data or large-capacity data can be placed in DRAM. The mathematical optimization model and orchestration method of this invention are also applicable to this scenario. By simply defining different memory layers as different "memory regions" and configuring them with corresponding memory access characteristic information (such as access distance, bandwidth contention coefficient, etc.), the optimal cross-layer data placement and orchestration scheme can be solved.

[0057] To accurately quantify the impact of different data memory placement combinations on system performance, a quantitative calculation method for memory access costs can be established. For example, in one implementation, step S12, obtaining the memory access cost corresponding to each of the data memory placement combinations, may include: obtaining the memory access cost corresponding to each of the data memory placement combinations based on the memory access feature information and preset weight information corresponding to the memory region.

[0058] This embodiment calculates the memory access cost for each data memory placement combination by using memory region access characteristic information and preset weight information. The memory access characteristic information can be a multi-dimensional parameter set describing the performance and overhead attributes of a specific memory region under a specific access scenario. The preset weight information is a set of coefficients used to characterize the relative importance of different memory access characteristic factors in the total cost calculation. Specifically, for any data memory placement combination, i.e., the pairing of data group g with memory region m, the corresponding memory access cost is not a single static value, but a comprehensive quantitative index obtained by weighting a series of dynamic and static memory access characteristic information of memory region m, combined with the preset weights corresponding to each characteristic information. This index is used to measure the overall resource cost and performance loss that the system may incur to satisfy relevant access requests after placing the data group in the memory region.

[0059] For example, a data memory placement combination might consist of "high-frequency data group A" and "remote memory region B." Its memory access characteristics could include: access frequency (e.g., 1000 times / second), access distance (e.g., physical latency of 250 nanoseconds), and current bandwidth contention factor (e.g., 0.8, indicating high load). Preset weighting information is set according to system optimization goals. For example, for latency-sensitive applications, the weight of "access distance" could be set to 0.5, while the weight of "bandwidth contention" could be set to 0.3. Based on this, when obtaining the memory access cost of this combination, the quantified values ​​of the above characteristics (e.g., 1000, 250, 0.8) are multiplied by their respective preset weights and then summed to obtain a value that reflects the potential overhead of the placement scheme. The larger this value, the higher the total weighted access cost brought by the placement combination.

[0060] This embodiment introduces preset weight information to comprehensively evaluate multi-dimensional memory access characteristics, enabling the calculation of memory access costs to be adjusted differently according to system load. This prioritizes the memory access performance of critical tasks during the solution process, improving the accuracy and flexibility of resource orchestration.

[0061] In the above embodiments, the memory access characteristic information can cover a variety of dimensions that affect memory access efficiency. For example, in one embodiment, the memory access characteristic information corresponding to the memory region may include at least one of the following: access frequency, access distance, task weight, scheduling overhead, cache hit rate, bandwidth contention coefficient, access mode coefficient, prefetch penalty coefficient, power consumption coefficient, fragmentation coefficient, interconnect congestion coefficient, and thermal throttling coefficient.

[0062] This embodiment defines a set of memory access characteristic information for quantifying memory access costs. This set aims to comprehensively capture the nonlinear coupling factors affecting memory access performance under NUMA architecture. Specifically: Access frequency refers to the total number of times the core reads and writes to a specific data group; it can be obtained through performance monitoring unit (PMU) sampling, performance profiling tool perf, and hardware tracing. Access distance refers to the physical latency from the core to the target memory region; it can be obtained through memory latency tools or numadistance data. Task weight refers to the priority of running a task. For example, real-time tasks have a higher weight than background tasks. It can be obtained through CFS (Completely Fair Scheduler) priority, QoS (Quality of Service) labels, schedulers, etc. Cache hit rate refers to the probability that data is successfully accessed in each level of cache (L1, L2, L3 / last-level cache LLC); it can be obtained through the hit rate table for each core and each group. Access patterns are used to distinguish between sequential and random access, the latter of which typically incurs a higher latency penalty; for example, random / sequential ratio × hardware prefetch friendliness. Prefetch penalty refers to the bandwidth and power waste caused by invalid prefetching or cross-NUMA prefetching; this coefficient can be obtained through hardware prefetcher hit rate statistics. Bandwidth contention refers to the contention effect that occurs when multiple cores simultaneously access the same memory controller; this coefficient can be obtained through real-time readings from a bandwidth monitor. Interconnection congestion refers to the current load level of interconnect links between NUMA nodes (such as Infinity Fabric); this coefficient can be obtained through link load monitoring. The power consumption factor refers to the indirect impact of core frequency changes caused by Dynamic Voltage and Frequency Scaling (DVFS) on access latency; it can be obtained through the current P-state × power consumption model. Thermal throttling refers to the performance penalty caused by a memory region or related core triggering frequency reduction protection due to excessive temperature; it can be obtained through temperature sensors and thermal design power (TDP) budget. Scheduling overhead refers to the cost of cache misses and memory re-accesses caused by task migration or switching; it can be obtained through a migration cost model and the expected number of switching. The fragmentation factor refers to the degree to which memory regions fail to allocate large pages or increase allocator overhead due to page fragmentation; it can be obtained by multiplying the hugepage failure rate by the allocator overhead (malloc / free trace).

[0063] The aforementioned features can be used individually or in combination to construct memory access cost models with varying degrees of precision and complexity. For example, for a bandwidth-sensitive high-performance computing application, the memory access cost calculation can primarily include two features: "bandwidth contention" and "interconnect congestion"; while for a database application, more attention should be paid to "access distance," "cache hit rate," and "access pattern."

[0064] In practice, the above 12 factors can be quantified. For example, the effective delay multiplier can be denoted as... : Correspondingly, the total memory access cost of pairing data set g with memory region m It can be represented as: in, The task weight coefficient for CPU core c. This represents the frequency of CPU core c accessing memory data group g. This represents the access distance from kernel c to memory region m. For cache hit rate, For cache miss rate, For access mode coefficients, Pre-emptive punishment This is the bandwidth contention factor. This represents the interconnection congestion coefficient. The power consumption coefficient is... coefficient, Fragmentation coefficient.

[0065] In this way, when a group of tasks is scheduled to run on the core set c, and its memory data is located in region m, it can represent the total cost it incurs. This embodiment constructs a quantitative weighted memory access cost model that includes the above-mentioned multi-dimensional factors, enabling the evaluation of memory access overhead to cover a variety of real physical effects, from microscopic cache hits and bandwidth contention to macroscopic interconnect congestion and hot throttling. This results in a more realistic cost evaluation than the traditional model that only relies on "access distance".

[0066] In the embodiments provided by this invention, preset constraints can be applied to the solution process to ensure that the obtained placement scheme is feasible in the actual system and does not violate hardware limitations. That is, in one embodiment, the process of solving the cost function in step S14 may include: solving the cost function based on predefined constraints so that the target placement arrangement scheme satisfies the constraints.

[0067] This embodiment differs from traditional approaches that rely solely on heuristic rules or objective functions for unrestricted optimization. Instead, it introduces predefined constraints when solving the cost function. These constraints represent hard limitations imposed by real physical resources and system logic on the memory placement scheme. By imposing these constraints on the solution process, it ensures that the final target placement orchestration scheme not only theoretically meets predetermined requirements for total memory access cost (e.g., minimum or low), but is also physically and logically deployable and operable. For example, without capacity constraints, the solution algorithm might allocate a huge data set to an insufficient memory region, leading to a memory overflow error during actual deployment.

[0068] This implementation ensures that the optimization process always takes place within the feasible solution space by incorporating constraints as an inherent part of the solution process. As a result, the output target placement and arrangement scheme naturally satisfies all preset physical and logical constraints, avoiding the problem of unusable optimization results.

[0069] Specifically, the aforementioned constraints may include multiple hard constraints such as unique assignment, capacity limits, and allowed domains. For example, in one implementation, the constraints may include at least one of the following: unique assignment constraint, capacity constraint, and allowed domain constraint; wherein, the unique assignment constraint stipulates that each data block can only be placed in one memory region, the capacity constraint stipulates that the space occupied by the data block placed in each memory region cannot exceed the memory capacity of the memory region, and the allowed domain constraint stipulates the memory regions where the data block is allowed to be placed. For example, some memory regions are reserved for use by the operating system, and application data is not allowed to be placed in these regions.

[0070] This embodiment further defines three core constraints to ensure the feasibility of the memory placement orchestration scheme: unique assignment, capacity limit, and allowed domain limit. Specifically: Unique assignment constraint is defined as a constraint on a set of data. For each data block k, the total number of different memory regions m to which it is allocated must be 1. The logic behind this constraint is that a single data block cannot physically exist completely in two different memory regions simultaneously; it can only be placed in one memory region, thus avoiding logical ambiguity. Its formula is expressed as:

[0071] The capacity constraint is defined as follows: the sum of the memory volumes of all data blocks allocated to the same memory region m cannot exceed the available physical capacity of region m. This constraint directly maps to the finiteness of physical memory, preventing allocation failures caused by memory over-allocation. Its formula is expressed as:

[0072] in, Let k be the memory size of the dataset. This represents the available capacity of memory region m.

[0073] The allowable domain constraint is defined as whether a data block k can be placed into memory region m, subject to a predefined allowable matrix. The elements of this matrix take values ​​of 1 or 0, where 1 indicates placement is allowed and 0 indicates placement is prohibited. This constraint is used to handle scenarios such as operating system-reserved memory areas, specific hardware limitations, or user-defined policies. Its formula is expressed as:

[0074] in, =1 indicates that it is allowed to be placed. =0 indicates that it is prohibited to place.

[0075] This embodiment introduces constraints during the solution process, limiting the originally unrestricted search space to a feasible domain, thus ensuring the legality and usability of the solution results.

[0076] In this embodiment of the invention, after constructing the cost function, the cost function can be specifically solved. For example, in one implementation, the cost function is constructed as the sum of the products of the memory access cost corresponding to each of the data memory placement combinations and a predetermined decision variable, where the independent variable is the decision variable and the dependent variable, i.e., the function value, is the total memory access cost of the placement orchestration scheme. The decision variable is used to decide the selected data memory placement combination in the placement orchestration scheme, that is, to decide whether the memory access cost corresponding to each data memory placement combination is included in the total memory access cost. If the combination is selected, the memory access cost of the combination is included in the total memory access cost. Different selected combinations result in different total memory access costs. Then, in step S14, the process of solving the cost function based on predefined constraints may include: solving the cost function based on predefined constraints with the goal of minimizing the function value to obtain the decision variable value; and obtaining the target placement orchestration scheme based on the obtained decision variable value.

[0077] Specifically, transforming the memory orchestration problem into a standard mathematical optimization problem and solving it can include the following process: S101. Construct the cost function.

[0078] The total memory access cost function F(x) is constructed as follows:

[0079] Among them, the data set to be placed into memory resources It is the memory access cost of placing the pre-calculated data block k in memory region m; This is the decision variable to be solved. The value of this variable determines whether the target placement and arrangement scheme includes the combination of data block k and memory region m. Whether it is included in the total cost. There are two possible solutions. A solution of 1 indicates that the target placement orchestration includes the combination of data block k and memory region m, meaning that data block k is specified to be placed in memory region m. A solution of 0 indicates that the target placement orchestration does not include the combination of data block k and memory region m. Clearly, if the value of the variable ensures that the target placement orchestration includes the combination of data block k and memory region m, then the corresponding... The total memory access cost included in the target placement orchestration scheme.

[0080] S102. Define the solution objective. Based on predefined constraints of unique assignment, capacity, and allowed domain, construct an optimization problem with the objective of minimizing the total memory access cost function F(x).

[0081] For example, the constraints in the solution process can be expressed as:

[0082]

[0083]

[0084] S103. Solve for the decision variables. Under the premise of satisfying all constraints, solve for the objective function F(x) to find the set of optimal decision variable values ​​that minimize F(x). .

[0085] S104. Generate an orchestration scheme. Based on the optimal decision variable values ​​obtained in step S103, determine the target memory region m (i.e., all) where each data block k should be placed. =1 corresponds to k,m), thus forming a complete target placement arrangement scheme.

[0086] This embodiment integrates the discrete decision-making process of "whether to place" into a continuous objective function by introducing 0-1 decision variables and expressing the total cost as the sum of its products and individual costs. For scenarios that require selecting the optimal subset from a massive number of candidate combinations, this embodiment uses explicit mathematical modeling to allow the problem to be solved directly using classic algorithms from operations research, thereby efficiently identifying the precise placement scheme that minimizes the total memory access cost.

[0087] For large-scale problem instances, efficient solution strategies can be employed to balance solution quality and computation time. For example, in one implementation, solving the cost function based on predefined constraints to obtain the decision variable values ​​may include: First, a predetermined heuristic construction algorithm is used to generate an initial feasible solution for the cost function; Then, based on the initial feasible solution, a metaheuristic global search algorithm is used to perform iterative search processing in the solution space of the cost function to obtain optimized candidate solutions whose function values ​​are lower than the function values ​​of the cost function corresponding to the initial feasible solution. Then, based on the optimized candidate solution, the values ​​of the decision variables are obtained.

[0088] This embodiment achieves hybrid solution through two stages: rapid startup (generating an initial feasible solution) and intelligent exploration (obtaining optimized candidate solutions). For example... Figure 2 As shown, the hybrid solution may include the following steps: S301, Rapid Startup Phase. One or more pre-defined heuristic construction algorithms are used to construct one or more initial feasible solutions that satisfy all constraints in a very short time. For example, the predetermined heuristic construction algorithm is an optimally adapted greedy algorithm. The total memory access cost corresponding to the initial feasible solution is usually high, providing an effective search starting point for subsequent optimization.

[0089] S302, Intelligent Exploration Phase. Starting from the initial feasible solution generated in step S301, a metaheuristic global search algorithm is used to conduct extensive searches and iterations in the solution space; Metaheuristic global search algorithms, by introducing mechanisms such as a certain probability of accepting inferior solutions, can effectively escape local optima traps and thus explore regions better than the initial solution. For example, metaheuristic global search algorithms can choose simulated annealing or genetic algorithms.

[0090] S303. Iteratively execute step S302 until the preset stopping conditions are met (such as the number of iterations, running time, or the quality of the solution no longer improves), and obtain an optimized candidate solution whose total memory access cost function value is significantly lower than the initial feasible solution.

[0091] S304. Based on this candidate optimization solution, the corresponding decision variable values ​​are analyzed (i.e., The value of can be used as input for the next stage (e.g., the exact convergence stage) or directly as the basis for the final decision.

[0092] This embodiment overcomes the contradiction between efficiency and global optimality of a single algorithm by adopting a phased hybrid solution strategy of "rapid start-up - intelligent exploration". For large-scale orchestration problems involving hundreds or thousands of data sets and dozens of memory regions, this embodiment first uses a heuristic algorithm to quickly obtain a feasible solution as a "seed", and then uses a metaheuristic algorithm (such as simulated annealing) to escape local optima traps in the vast solution space, thereby approaching the global optimal solution in an acceptable time (milliseconds), balancing the solution speed and the quality of the solution.

[0093] To ensure strict convergence in the pursuit of the theoretically optimal solution, an exact algorithm can be further employed based on the aforementioned candidate solutions. For example, in one implementation, obtaining the decision variable values ​​based on the candidate solutions may include: pruning the cost function corresponding to the candidate solutions using a predetermined branch-and-bound algorithm, with the cost function value as a global upper bound, to obtain a target optimal solution where the cost function value is lower than the cost function value corresponding to the candidate solutions; and obtaining the decision variable values ​​based on the target optimal solution.

[0094] Unlike heuristic or metaheuristic algorithms that can only find a "better solution", this embodiment aims to theoretically guarantee the "optimality" of the solution. After obtaining a high-quality candidate solution, this embodiment initiates the precise convergence phase.

[0095] Specifically, the Branch and Bound (BnB) algorithm is employed, and the total memory access cost function value corresponding to the optimized candidate solution is set as a global upper bound. The BnB algorithm systematically searches the solution space and calculates the lower bound of the optimal cost that each partial solution (branch) can reach. When the lower bound is greater than or equal to the currently known global upper bound, that branch and all its sub-branches necessarily do not contain better solutions and are thus "pruned." Because the high-quality solutions obtained in the previous stage are used as the upper bound, a large number of non-optimal branches can be eliminated early, greatly narrowing the search range. This allows the algorithm to accurately find the target global optimal solution whose total memory access cost function value is strictly lower than the original candidate solution within an acceptable time cost. Finally, the decision variable values ​​derived from this global optimal solution have a strict mathematical optimality guarantee.

[0096] This embodiment, based on the high-quality solutions obtained in the preliminary stages, introduces a branch and bound algorithm for precise convergence. For scenarios seeking theoretical absolute optimality or reducing the problem size, this embodiment uses the known high-quality solutions as a "benchmark" (global upper bound). During the search process, all branches that cause the total cost to exceed this benchmark are directly pruned, greatly compressing the search space. This allows for precise locking of the global optimal solution within a controllable time, providing theoretical assurance for scenarios with extremely demanding performance requirements.

[0097] In practice, a greedy construction strategy can be used to generate an initial feasible solution to speed up the solution process. For example, in one implementation, generating an initial feasible solution for the cost function using a predetermined heuristic construction algorithm may include: sorting all the data groups in descending order of memory usage; sequentially allocating the allowable memory region with the smallest remaining capacity after placing the data group in each of the descendingly sorted data groups, and updating the available capacity of the selected memory region; and obtaining the initial feasible solution based on the allocation scheme formed after placing all the data groups.

[0098] This implementation provides a specific greedy algorithm for generating initial feasible solutions, using a best-fit strategy sorted in descending order of memory usage, such as... Figure 3 As shown, the steps may include: S401. Obtain the set of all data blocks to be placed, and obtain the set of all available memory regions and their capacities. ; S402. Sort all data groups in the set according to their memory usage. Sort the data in descending order from largest to smallest to form a queue of pending processes; S403. Initialize a list of remaining capacity for memory regions, with initial values ​​for each region. ; S404. Take the first (i.e. the largest) data group k from the queue to be processed; S405. Traverse all memory regions that are allowed to hold the data set k (based on the allowed domain constraints). =1), select a region m from which the current remaining capacity of region m is greater than or equal to 1. And after placement, the remaining capacity of this area is the smallest among all areas that meet the conditions; S406. Allocate data group k to the selected memory region m, record this allocation, and reduce the remaining capacity of region m. ; S407. Repeat steps S404 to S406 until the queue to be processed is empty; S408. Once all data groups have been allocated, a complete placement scheme is formed based on all allocation records. This scheme is an initial feasible solution that satisfies the capacity constraint and the allowable domain constraint.

[0099] This embodiment employs a greedy strategy of "large object priority" and "best fit" to make memory regions more compact, enabling the rapid generation of initial placement schemes with low memory fragmentation. For scenarios with limited memory resources or frequent large page allocations, this greedy construction can generate a feasible solution with high memory utilization and low fragmentation in an extremely short time (microseconds), providing a high-quality starting point for subsequent intelligent exploration phases.

[0100] To avoid local optima bias caused by a single greedy strategy, multiple greedy algorithms with different rules can be executed in parallel to increase the diversity of initial solutions. For example, in one implementation, generating initial feasible solutions for the cost function using a predetermined heuristic construction algorithm may include: identifying at least two greedy algorithm variants, wherein the sorting rules or selection criteria of the at least two greedy algorithm variants differ; executing each of the greedy algorithm variants in parallel, sorting all the data groups according to the sorting rules specified by the variant; sequentially allocating memory regions for each of the sorted data groups according to the selection criteria specified by the variant, and updating the available capacity of the selected memory regions; collecting the allocation schemes obtained from the parallel execution of all the greedy algorithm variants to obtain multiple initial feasible solutions.

[0101] This embodiment overcomes the tendency of a single algorithm to get trapped in local optima by running multiple greedy algorithm variants with different configurations in parallel. Specifically, this implementation predefines at least two greedy algorithm variants that differ in their "sorting rules" (e.g., descending or ascending order by memory usage) or "memory region selection criteria" (e.g., selecting the region with the smallest or largest remaining capacity after placement). These variants are executed independently in parallel. Each variant sorts and allocates the data set according to its own unique rules, ultimately generating initial feasible solutions with different structures.

[0102] This embodiment generates multiple high-quality initial feasible solutions with different structures and distributed in different regions by running multiple greedy algorithms with different rules in parallel. This avoids the "local optimum trap" caused by a single greedy strategy, significantly increases the coverage of the solution space by the initial solutions, and provides a richer "seed" population for the next stage of intelligent search, thereby increasing the probability of finding the global optimum.

[0103] In the parallel greedy algorithm construction process, sorting rules and selection criteria can be combined in various flexible ways. For example, in one implementation, the sorting rules may include at least one of the following: descending order of memory usage, ascending order of memory usage, descending order of access frequency, and descending order of task weight; and / or the selection criteria may include at least one of the following: selecting the option with the smallest remaining capacity after placement, selecting the option with the largest remaining capacity after placement, selecting the option with the smallest access latency, and selecting the option with the largest currently available capacity.

[0104] This embodiment specifically defines various sorting rules and memory region selection criteria for constructing different greedy algorithm variants. The sorting rules, used to determine the order in which data groups are processed, include, but are not limited to: Memory usage in descending order: Prioritize processing the data group that uses the most memory; Memory usage in ascending order: Prioritize processing the data group that uses the least memory; Access frequency in descending order: Prioritize processing the data group that is accessed most frequently by the core. Task weight descending order: Prioritize processing data groups associated with high-priority tasks.

[0105] The memory region selection criteria are used to select from all allowed candidate regions when allocating a data group, including but not limited to: The optimal adaptation strategy, which minimizes the remaining capacity after placement, aims to maximize memory region utilization and reduce fragmentation. Choosing the option that maximizes the remaining capacity after placement: This is the worst-case adaptation strategy, designed to reserve space for subsequent large objects. Select the memory region with the lowest access latency: Prioritize allocating data groups to the memory region that is physically closest (with the lowest access latency); Select the region with the largest available capacity: Prioritize allocating data groups to the region with the most free space to simplify the allocation process.

[0106] By combining the different sorting rules and selection criteria mentioned above, a large number of greedy algorithm variants with different behaviors can be derived. For example, combining "descending order of memory usage" with "selecting the option with the minimum remaining capacity after placement" yields the aforementioned best-fit descending order algorithm; combining "descending order of access frequency" with "selecting the option with the minimum access latency" results in a greedy algorithm that focuses on providing fast access to hot data.

[0107] This embodiment provides configurable sorting rules and selection criteria, enabling the greedy algorithm to adapt its behavior to different optimization objectives. This prioritizes the low-latency placement of high-priority data blocks, allowing for the generation of an initial solution with a clear performance orientation even during the rapid startup phase.

[0108] To efficiently escape local optima in a large solution space, a parallel simulated annealing algorithm can be used for global search. For example, in one implementation, the metaheuristic global search algorithm includes a parallel simulated annealing algorithm. The iterative search process in the solution space of the cost function based on the initial feasible solution can include: starting from the initial feasible solution, exploring multiple neighborhoods or temperature states in parallel to obtain optimized candidate solutions where the cost function value is lower than the cost function value corresponding to the initial feasible solution.

[0109] To efficiently find better candidate solutions starting from the initial feasible solution, this implementation provides a specific process for intelligent exploration using a parallel simulated annealing algorithm, which may include: S501. Initialization. Using one or more initial feasible solutions as the starting point for the search, set the initial temperature (high temperature), cooling rate, number of iterations at each temperature (Markov chain length), and termination temperature for the simulated annealing algorithm.

[0110] S502, Parallel Search. During the execution of the algorithm, parallel computing capabilities are utilized to simultaneously explore the neighborhoods of multiple solution spaces or run multiple annealing chains at different temperature states.

[0111] For example, one processing core is responsible for performing a neighborhood perturbation on the current solution by "swapping the memory regions of two data sets" and calculating the cost of the new solution; another processing core may be responsible for performing a perturbation on a solution with a different structure by "moving a data set to another region".

[0112] S503. Probabilistic Acceptance. For each new solution generated by parallel exploration, according to the Metropolis criterion: if the cost function value of the new solution is lower than that of the current solution, the new solution is accepted unconditionally; if the cost of the new solution is higher, the inferior solution is accepted with a probability related to the difference between the current temperature and the cost. This helps to escape local optima.

[0113] S504, Iteration and Cooling. After completing the parallel exploration at a temperature, the temperature is reduced according to the cooling strategy, and steps S502 and S503 are repeated at the new temperature.

[0114] S505. Obtaining Optimization Candidate Solutions. When the temperature drops to the termination temperature or the quality of the solution does not significantly improve at multiple consecutive temperatures, the algorithm terminates and records the solution with the lowest total memory access cost function encountered during the search process as an optimization candidate solution. This candidate solution is significantly better than the initial feasible solution.

[0115] This embodiment employs a parallel simulated annealing algorithm, enabling simultaneous exploration of multiple search paths within the solution space. For complex orchestration problems with a vast search space and multiple local optima, this scheme utilizes a parallel mechanism to accelerate the neighborhood search and annealing process. This not only significantly shortens the time to find a high-quality solution but also greatly reduces the risk of getting trapped in undesirable local optima during the cooling process through multi-state parallelism, thereby converging to a globally near-optimal solution more efficiently.

[0116] Furthermore, in this embodiment of the invention, after placing data blocks into memory regions based on a target placement orchestration scheme, possible changes in the runtime environment can be monitored in real time, and adaptive adjustments can be made. For example, in one embodiment, after placing the data blocks into the memory regions based on the obtained target placement orchestration scheme, the method may further include: real-time monitoring of memory access performance indicators for each data group in the memory region, wherein the memory access performance indicators may include at least one of the following: average memory access latency, remote access ratio, memory bandwidth utilization, and cache hit rate; if the memory access performance indicators do not meet preset conditions, key parameters are selected from the memory access quantification information of the memory regions where the data groups are placed, and the weight values ​​corresponding to each key parameter are adjusted; based on the key parameters and the adjusted weight values, the memory access cost of the data memory placement combination is re-obtained; then, the cost function is constructed and solved using the re-obtained memory access cost to obtain an updated target placement orchestration scheme; based on the updated target placement orchestration scheme, the data blocks are placed into the updated memory regions until the memory access performance indicators meet preset conditions.

[0117] This implementation introduces a closed-loop dynamic optimization process after the initial placement, enabling memory orchestration to adapt to runtime load changes. The specific data flow and processing flow are as follows: Data Acquisition: The system's monitoring module continuously collects data in real time from the hardware performance counter (PMU), operating system, or runtime environment. Collected memory access performance metrics include, but are not limited to: average memory access latency, remote access ratio, memory bandwidth utilization, and cache hit rate.

[0118] Trigger Judgment: The decision module receives the above performance indicator data and compares it with a preset threshold (i.e., preset condition). For example, the preset condition is "remote access ratio is less than 20%". If the current "remote access ratio" is higher than 20%, the preset condition is not met, and the re-optimization process is triggered.

[0119] Model Adjustment: After triggering re-optimization, the system analyzes performance bottlenecks and selects the key parameters most relevant to the current performance bottleneck from the multi-dimensional feature information constituting the memory access cost model. For example, if "memory bandwidth utilization" is too high, then "bandwidth contention" and "interconnect congestion" are selected as key parameters. Subsequently, the system dynamically increases the weight values ​​of these key parameters in the cost function according to predefined rules or by using machine learning methods.

[0120] Re-optimization and deployment: Using the adjusted weight values ​​and selected key parameters, re-execute the aforementioned memory access cost calculation, cost function construction, and solution process (potentially only for the affected local data set or fast incremental algorithm) to generate an updated target placement orchestration scheme aimed at alleviating the current bottleneck. Finally, based on this scheme, migrate the relevant data blocks to the new target memory region.

[0121] Loop: After the migration is complete, the system continues to monitor performance metrics and repeats the above process until all metrics meet the preset conditions and the system reaches a new stable and optimal performance state.

[0122] This embodiment can proactively trigger weight adjustment and re-solution processes by monitoring performance degradation indicators such as latency or bandwidth in real time. This allows data to be dynamically migrated to a better memory area without interrupting service, achieving adaptive response to load phase changes and keeping the system in an optimal performance state for a long time.

[0123] The memory resource arrangement method provided by the embodiments of the present invention will be described in detail below through specific examples.

[0124] like Figure 4 As shown, the memory resource orchestration method provided in the embodiments of the present invention may include: S601. Based on a data set to be placed into a memory region and at least one memory region, determine a data memory placement combination, wherein the data set includes at least one data block, and the data memory placement combination includes the data block and a memory region to which the data block is to be placed; S602. Based on the memory access feature information and preset weight information corresponding to the memory region, obtain the memory access cost corresponding to each data memory placement combination; Optionally, the memory access characteristic information corresponding to the memory region includes at least one of the following: access frequency, access distance, task weight, scheduling overhead, cache hit rate, bandwidth contention, access mode, prefetch penalty, power consumption coefficient, fragmentation coefficient, interconnect congestion, and hot throttling; S603. Based on the memory access cost corresponding to each data memory placement combination, construct a cost function to characterize the total memory access cost corresponding to the placement orchestration scheme, wherein the total memory access cost corresponding to the placement orchestration scheme is the sum of the memory access costs corresponding to each data memory placement combination in the placement orchestration scheme. S604. The cost function is constructed as the sum of the product of the memory access cost corresponding to each data memory placement combination and a predetermined decision variable, wherein the decision variable is used to decide whether the memory access cost corresponding to the data memory placement combination is included in the total memory access cost. S605. Sort all data groups in descending order of memory usage, and for each data group in the descending order, prioritize the allocation of the allowable memory region with the smallest remaining capacity after placing the data group, and update the available capacity of the selected memory region. S606. Based on the allocation scheme formed after all data groups have been placed, obtain an initial feasible solution; S607. Based on the initial feasible solution, a metaheuristic global search algorithm is used to perform iterative search in the solution space of the cost function to obtain optimized candidate solutions whose cost function values ​​are lower than those of the initial feasible solution. S608. Based on the optimized candidate solution, obtain the value of the decision variable; S609. Based on the obtained decision variable values, obtain the target placement and arrangement scheme; S610. Based on the acquired target placement arrangement scheme, place the data blocks into the corresponding memory area.

[0125] Secondly, embodiments of the present invention provide a memory resource orchestration apparatus that effectively improves the access efficiency of memory resources.

[0126] like Figure 5 As shown, the memory resource orchestration apparatus provided in the embodiments of the present invention includes at least one memory region, and the apparatus may include: The determining unit 31 is configured to determine a data memory placement combination based on the data set to be placed into the memory resource and the at least one memory region, wherein the data set includes at least one data block, and the data memory placement combination includes the data block and the memory region to which the data block is to be placed; The acquisition unit 32 is used to acquire the memory access cost corresponding to each of the data memory placement combinations; Construction unit 33 is used to construct a cost function to characterize the total memory access cost corresponding to the placement orchestration scheme based on the memory access cost corresponding to each data memory placement combination, wherein the total memory access cost corresponding to the placement orchestration scheme is the sum of the memory access costs corresponding to each data memory placement combination in the placement orchestration scheme; The solving unit 34 is used to solve the cost function to obtain a target placement arrangement scheme, wherein the target placement arrangement scheme makes the function value of the cost function meet predetermined requirements; The placement unit 35 is used to place the data block into the memory area according to the obtained target placement arrangement scheme.

[0127] The memory resource orchestration apparatus provided in the embodiments of the present invention can determine data memory placement combinations based on the data set to be placed in the memory region and the memory region, obtain the memory access cost corresponding to each data memory placement combination, construct a cost function to characterize the total memory access cost of the placement orchestration scheme based on each memory access cost, solve the cost function to obtain a target placement orchestration scheme that makes the cost function value meet predetermined requirements, and place data blocks into the memory region accordingly based on the target placement orchestration scheme. In this way, a cost function to characterize the total memory access cost corresponding to the placement orchestration scheme can be constructed based on the memory access cost corresponding to each data memory placement combination, formalizing the memory orchestration problem into a solvable mathematical model, and performing global optimization based on the quantized memory access cost. Therefore, it can systematically reduce cross-node remote access latency and bandwidth contention, reduce unnecessary page migration and cache invalidation overhead, and thus effectively improve the access efficiency of memory resources.

[0128] In one embodiment, the acquisition unit 32 is specifically used to: acquire the memory access cost corresponding to each data memory placement combination based on the memory access feature information and preset weight information corresponding to the memory region.

[0129] In one embodiment, the memory access characteristic information corresponding to the memory region includes at least one of the following: access frequency, access distance, task weight, scheduling overhead, cache hit rate, bandwidth contention, access mode, prefetch penalty, power consumption coefficient, fragmentation coefficient, interconnect congestion, and hot throttling.

[0130] In one embodiment, the solving unit 34 includes a solving module for solving the cost function based on predefined constraints, so that the target placement arrangement scheme satisfies the constraints.

[0131] In one implementation, the constraints include at least one of the following: unique assignment constraint, capacity constraint, and allowable domain constraint; wherein the unique assignment constraint stipulates that each data block can only be placed in one memory region, the capacity constraint stipulates that the space occupied by the data block placed in each memory region cannot exceed the memory capacity of the memory region, and the allowable domain constraint stipulates the memory regions where the data block is allowed to be placed.

[0132] In one implementation, the cost function is constructed as the sum of the products of the memory access cost corresponding to each of the data memory placement combinations and a predetermined decision variable, wherein the decision variable is used to decide whether the memory access cost corresponding to the data memory placement combination is included in the total memory access cost; the solution module is specifically used to: solve the cost function based on predefined constraints, with the goal of minimizing the function value, to obtain the value of the decision variable; and obtain the target placement orchestration scheme based on the obtained decision variable value.

[0133] In one implementation, the solution module is specifically used to: generate an initial feasible solution for the cost function using a predetermined heuristic construction algorithm; perform iterative search processing in the solution space of the cost function based on the initial feasible solution using a metaheuristic global search algorithm to obtain optimized candidate solutions where the function value of the cost function is lower than the function value of the cost function corresponding to the initial feasible solution; and obtain the decision variable value based on the optimized candidate solution.

[0134] In one implementation, the solution module is specifically used to: prune the cost function corresponding to the candidate optimization solution using a predetermined branch and bound algorithm, taking the function value of the cost function as the global upper bound, to obtain a target optimal solution where the function value of the cost function is lower than the function value of the cost function corresponding to the candidate optimization solution; and obtain the decision variable value based on the target optimal solution.

[0135] In one implementation, the solution module is specifically used to: sort all the data groups in descending order of memory usage; sequentially select the allowable memory region with the smallest remaining capacity after placing the data group in the descending order of the data groups, and update the available capacity of the selected memory region; and obtain the initial feasible solution based on the allocation scheme formed after placing all the data groups.

[0136] In one implementation, the solution module is specifically configured to: determine at least two greedy algorithm variants, wherein the sorting rules or selection criteria of the at least two greedy algorithm variants differ; execute each of the greedy algorithm variants in parallel, and sort all the data groups according to the sorting rules specified by the variants; sequentially select a memory region for allocation for each of the sorted data groups according to the selection criteria specified by the variants, and update the available capacity of the selected memory region; collect the allocation schemes obtained by executing all the greedy algorithm variants in parallel, and obtain multiple initial feasible solutions.

[0137] In one implementation, the sorting rules include at least one of the following: descending order of memory usage, ascending order of memory usage, descending order of access frequency, and descending order of task weight; and / or the selection criteria include at least one of the following: selecting the option with the smallest remaining capacity after placement, selecting the option with the largest remaining capacity after placement, selecting the option with the smallest access latency, and selecting the option with the largest currently available capacity.

[0138] In one implementation, the metaheuristic global search algorithm includes a parallel simulated annealing algorithm, and the solution module is specifically used to: take the initial feasible solution as a starting point, explore multiple neighborhoods or temperature states in parallel, and obtain optimized candidate solutions where the function value of the cost function is lower than the function value of the cost function corresponding to the initial feasible solution.

[0139] In one embodiment, the apparatus further includes: a monitoring unit, configured to monitor in real time the memory access performance indicators of each data group in the memory region after the data block is placed in the memory region according to the acquired target placement orchestration scheme, the memory access performance indicators including at least one of the following: average memory access latency, remote access ratio, memory bandwidth utilization, and cache hit rate; and an adjustment unit, configured to, when the memory access performance indicators do not meet preset conditions, select key parameters from the memory access quantification information of the memory region where the data group is placed, adjust the weight values ​​corresponding to each key parameter, re-acquire the memory access cost of the data memory placement combination based on the key parameters and the adjusted weight values, construct and solve the cost function through the re-acquired memory access cost to obtain an updated target placement orchestration scheme, and place the data block in the updated memory region according to the updated target placement orchestration scheme until the memory access performance indicators meet preset conditions.

[0140] In one embodiment, the memory resources include at least one of the following: local memory of a non-consistent memory access node, dynamic random access memory, high-bandwidth memory, persistent memory, at least two levels of storage in a hierarchical storage system, video memory of a graphics processor, extended memory devices or memory pools connected via a compute fast link protocol.

[0141] Thirdly, embodiments of the present invention also provide an electronic device capable of optimizing the data placement and arrangement of memory resources to reduce memory access costs.

[0142] like Figure 6 As shown, the electronic device provided in the embodiments of the present invention may include: a processor 71 and a memory 72, wherein the processor 71 and the memory 72 are electrically connected; the memory 72 is used to store executable program code; the processor 71 runs a program corresponding to the executable program code by reading the executable program code stored in the memory 72, so as to implement any of the memory resource arrangement methods provided in the foregoing embodiments.

[0143] The specific execution process of the above steps by the processor 71, as well as the steps further executed by the processor 71 by running executable program code, can be found in the description of the foregoing embodiments, and will not be repeated here.

[0144] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement any of the memory resource arrangement methods provided in the foregoing embodiments, thus achieving the corresponding technical effects. This has been described in detail above and will not be repeated here.

[0145] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0146] The various embodiments in this specification are described in a related manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0147] In particular, the device embodiment is basically similar to the method embodiment, so the description is relatively simple. For relevant details, please refer to the description of the method embodiment.

[0148] For ease of description, the above apparatus is described by dividing it into various functional units / modules. Of course, in implementing this invention, the functions of each unit / module can be implemented in one or more software and / or hardware.

[0149] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0150] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for arranging memory resources, wherein the memory resources include at least one memory region, characterized in that, include: Based on the data set to be placed into the memory resource and the at least one memory region, a data memory placement combination is determined, wherein the data set includes at least one data block, and the data memory placement combination includes the data block and the memory region to which the data block is to be placed; Obtain the memory access cost corresponding to each of the data memory placement combinations; Based on the memory access cost corresponding to each data memory placement combination, a cost function is constructed to characterize the total memory access cost corresponding to the placement orchestration scheme, wherein the total memory access cost corresponding to the placement orchestration scheme is the sum of the memory access costs corresponding to each data memory placement combination in the placement orchestration scheme; The cost function is solved to obtain a target placement arrangement scheme, which makes the function value of the cost function meet predetermined requirements. Based on the obtained target placement arrangement scheme, the data blocks are placed into the memory area accordingly.

2. The memory resource arrangement method according to claim 1, characterized in that, The memory access cost for obtaining each of the data memory placement combinations includes: Based on the memory access feature information and preset weight information corresponding to the memory region, the memory access cost corresponding to each data memory placement combination is obtained.

3. The memory resource arrangement method according to claim 2, characterized in that, The memory access characteristic information corresponding to the memory region includes at least one of the following: access frequency, access distance, task weight, scheduling overhead, cache hit rate, bandwidth contention, access mode, prefetch penalty, power consumption coefficient, fragmentation coefficient, interconnect congestion, and hot throttling.

4. The memory resource arrangement method according to claim 1, characterized in that, The process of solving the cost function includes: Based on predefined constraints, the cost function is solved to ensure that the target placement and arrangement scheme satisfies the constraints.

5. The memory resource arrangement method according to claim 4, characterized in that, The constraints include at least one of the following: unique assignment constraint, capacity constraint, and allowable field constraint; wherein, the unique assignment constraint stipulates that each data block can only be placed in one memory region, the capacity constraint stipulates that the space occupied by the data block placed in each memory region cannot exceed the memory capacity of the memory region, and the allowable field constraint stipulates the memory regions that are allowed to place the data block.

6. The memory resource arrangement method according to claim 4, characterized in that, The cost function is constructed as the sum of the product of the memory access cost corresponding to each of the data memory placement combinations and a predetermined decision variable, wherein the decision variable is used to decide whether the memory access cost corresponding to the data memory placement combination is included in the total memory access cost. The process of solving the cost function based on predefined constraints includes: Based on predefined constraints, the cost function is solved with the goal of minimizing the function value to obtain the decision variable values. Based on the obtained decision variable values, a target placement and arrangement scheme is obtained.

7. The memory resource arrangement method according to claim 6, characterized in that, The process of solving the cost function based on predefined constraints to obtain the values ​​of the decision variables includes: An initial feasible solution to the cost function is generated using a predetermined heuristic construction algorithm. Based on the initial feasible solution, a metaheuristic global search algorithm is used to perform iterative search processing in the solution space of the cost function to obtain optimized candidate solutions whose function values ​​are lower than those of the initial feasible solution. Based on the optimized candidate solution, the values ​​of the decision variables are obtained.

8. The memory resource arrangement method according to claim 7, characterized in that, The process of obtaining the decision variable values ​​based on the optimized candidate solution includes: By using a predetermined branch and bound algorithm, the function value of the cost function corresponding to the candidate optimization solution is used as the global upper bound for pruning, so as to obtain the target optimal solution whose function value of the cost function is lower than the function value of the cost function corresponding to the candidate optimization solution; Based on the target optimal solution, the values ​​of the decision variables are obtained.

9. The memory resource arrangement method according to claim 7, characterized in that, The process of generating an initial feasible solution to the cost function using a predetermined heuristic construction algorithm includes: Sort all the data groups in descending order of memory usage; For each data group in the descending sorted data group, the allowable memory region with the smallest remaining capacity after placing the data group is selected for allocation, and the available capacity of the selected memory region is updated. Based on the allocation scheme formed after all the data groups have been placed, the initial feasible solution is obtained.

10. The memory resource arrangement method according to claim 7, characterized in that, The process of generating an initial feasible solution to the cost function using a predetermined heuristic construction algorithm includes: Identify at least two greedy algorithm variants, wherein the sorting rules or selection criteria of the at least two greedy algorithm variants differ; Each of the greedy algorithm variants is executed in parallel, and all the data sets are sorted according to the sorting rules specified by the variant. For each of the sorted data groups in sequence, a memory region is selected for allocation according to the selection criteria specified by the variant, and the available capacity of the selected memory region is updated. Collect all the allocation schemes obtained by executing all the greedy algorithm variants in parallel to obtain multiple initial feasible solutions.

11. The memory resource arrangement method according to claim 10, characterized in that, The sorting rules include at least one of the following: descending order of memory usage, ascending order of memory usage, descending order of access frequency, and descending order of task weight; and / or The selection criteria include at least one of the following: selecting the option with the smallest remaining capacity after placement, selecting the option with the largest remaining capacity after placement, selecting the option with the smallest access latency, and selecting the option with the largest currently available capacity.

12. The memory resource arrangement method according to claim 7, characterized in that, The metaheuristic global search algorithm includes a parallel simulated annealing algorithm. The step of performing iterative search processing in the solution space of the cost function based on the initial feasible solution using a metaheuristic global search algorithm includes: Starting from the initial feasible solution, multiple neighborhoods or temperature states are explored in parallel to obtain optimized candidate solutions whose cost function values ​​are lower than the cost function values ​​corresponding to the initial feasible solution.

13. The memory resource arrangement method according to claim 2, characterized in that, After placing the data blocks into the memory region based on the obtained target placement orchestration scheme, the method further includes: Real-time monitoring of memory access performance metrics for each data group in the memory region, wherein the memory access performance metrics include at least one of the following: average memory access latency, remote access ratio, memory bandwidth utilization, and cache hit rate; If the memory access performance index does not meet the preset conditions, key parameters are selected from the memory access quantification information of the memory region where the data group is placed, and the weight values ​​corresponding to each key parameter are adjusted. Based on the key parameters and the adjusted weight values, the memory access cost of the data memory placement combination is re-acquired. Then, the cost function is constructed and solved using the re-acquired memory access cost to obtain an updated target placement orchestration scheme. Based on the updated target placement orchestration scheme, the data block is placed in the updated memory region until the memory access performance index meets the preset conditions.

14. The memory resource arrangement method according to claim 1, characterized in that, The memory resources include at least one of the following: local memory of non-uniform memory access nodes, dynamic random access memory, high-bandwidth memory, persistent memory, at least two levels of storage in a hierarchical storage system, video memory of a graphics processor, extended memory devices or memory pools connected via a compute fast link protocol.

15. A memory resource arrangement apparatus, wherein the memory resources include at least one memory region, characterized in that, include: A determining unit is configured to determine a data memory placement combination based on a data set to be placed into the memory resource and the at least one memory region, wherein the data set includes at least one data block, and the data memory placement combination includes the data block and a memory region to which the data block is to be placed; The acquisition unit is used to acquire the memory access cost corresponding to each of the data memory placement combinations; The construction unit is used to construct a cost function to characterize the total memory access cost corresponding to the placement orchestration scheme based on the memory access cost corresponding to each data memory placement combination, wherein the total memory access cost corresponding to the placement orchestration scheme is the sum of the memory access costs corresponding to each data memory placement combination in the placement orchestration scheme; The solution unit is used to solve the cost function to obtain a target placement arrangement scheme, wherein the target placement arrangement scheme makes the function value of the cost function meet predetermined requirements; The placement unit is used to place the data block into the memory area according to the obtained target placement arrangement scheme.

16. An electronic device, characterized in that, The electronic device includes: a processor and a memory, wherein the processor is electrically connected to the memory; the memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the memory resource arrangement method according to any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the memory resource arrangement method according to any one of claims 1 to 14.

Images