NUMA equalization method based on inter-process communication perception

Abstract

The invention relates to a non uniform memory access (NUMA) equalization method based on inter-process communication perception, which belongs to the technical field of computers and comprises the following steps: acquiring communication data of an internet protocol computer (IPC) and determining an affinity weight value of a node pair to which each process pair belongs; obtaining a load balancing factor value of each node; for any process, the node where the process is located is determined as a source node, other nodes are determined as target nodes, and the topological weight of each target node is determined according to the distance between the target node and the source node; determining the comprehensive score of each target node according to the affinity weight value, the topological weight and the load balancing factor value, and screening the target node with the highest comprehensive score as the optimal node of the source node; obtaining the migration income and migration cost of the optimal node, and judging whether the process on the source node is migrated to the optimal node or not; if the migration income is greater than or equal to 2 * migration cost, migration is carried out; and if the migration income is less than 2 * the migration cost, not migrating. The method has the effect of improving the overall performance of the system.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a NUMA load balancing method based on inter-process communication awareness. Background Technology

[0002] The current Linux kernel's automatic NUMA load balancing mechanism only analyzes process memory access patterns. It identifies memory access hotspots through page fault statistics, migrating frequently accessed memory pages to the node where the process resides, or relocating the process to a node with more frequently accessed memory. However, multi-core processor scheduling load balancing mechanisms migrate tasks between NUMA nodes to balance computational load. This can lead to frequently communicating processes being distributed across different nodes, increasing communication overhead and causing performance degradation. Specifically, the existing mechanism cannot identify inter-process communication relationships, considering only the memory access patterns of individual processes and ignoring the overall performance of process collaboration. Consequently, IPC communication needs to cross NUMA node boundaries, increasing latency and bandwidth consumption. Furthermore, data from related processes may be distributed across caches on multiple nodes, reducing cache hit rates and further degrading the performance of IPC communication processes, thus impacting overall system performance. Summary of the Invention

[0003] To improve the overall system performance, this application provides a NUMA load balancing method based on inter-process communication awareness.

[0004] A NUMA load balancing method based on inter-process communication awareness includes: Obtain IPC communication data; The communication frequency, communication delay duration, and amount of data transmitted between each process pair are determined based on the communication data. The communication density index is determined based on the communication frequency, communication delay duration, and data transmission volume, and the communication relationship level between each process pair is classified based on the communication density index. The affinity weight value of each process pair to its node pair is determined based on the communication relationship level between each process pair and the preset weight comparison table. Obtain the load balancing factor of each node, and obtain the load balancing factor weight value of each node according to the load balancing factor and the preset comparison relationship. Then determine the load balancing factor value of each node based on the load balancing factor weight value. For any given process, the node where it is located is determined as the source node, and the other nodes are determined as target nodes. Based on the distance between the target node and the source node, the topological weight of each target node is determined according to a preset topological factor lookup table. The comprehensive score of each target node is determined based on the affinity weight and topology weight of the target node relative to the source node, as well as the load balancing factor value of the target node itself, and the target node with the highest comprehensive score is selected as the optimal node of the source node. To determine the optimal node's migration benefits and costs; Determine whether to migrate processes from the source node to the optimal node based on migration benefits and migration costs. If the migration benefit is greater than or equal to 2 * migration cost, then the process on the source node will be migrated to the optimal node. If the migration benefit is less than 2 * migration cost, then the process on the source node will not be migrated to the optimal node.

[0005] Optionally, the step of classifying communication relationship levels based on the communication frequency, communication delay duration, and transmitted data volume includes: The communication frequency score is determined based on the communication frequency and the preset frequency score table; The communication delay score is determined based on the communication delay duration and the preset delay duration score table. The score for the transmitted data volume is determined based on the transmitted data volume and the preset data volume score table. The communication density index is determined based on the communication frequency score, communication delay duration score, data transmission volume score, and a preset weight table. The communication relationship level is determined based on the communication frequency, average delay duration, communication density index, and a preset relationship level comparison table.

[0006] Optionally, the load balancing factors for each node include CPU utilization, memory pressure, and task queue length load rate. The weight values ​​of the load balancing factors include CPU utilization weight value, memory pressure weight value, and task queue length load rate weight value.

[0007] Optionally, methods for obtaining CPU utilization weight values ​​include: Obtain the runtime and idle time of all CPU cores within each node; Calculate the sum of the running time and idle time of all CPU cores to obtain the total running time and total idle time; CPU utilization is determined based on the total running time and total idle time. The CPU utilization weight value is determined based on the CPU utilization and the preset CPU utilization comparison table.

[0008] Optional methods for obtaining memory pressure weight values ​​include: Get the used memory and total memory of the node; Memory utilization is determined based on the used memory and total memory. Obtain the system's page swapping frequency, i.e., the number of times a page is swapped in and out per unit of time; Memory pressure is determined based on memory utilization, page swapping frequency, preset memory utilization weight value, and preset page swapping frequency weight value. The memory pressure weight value is determined based on the memory pressure and the preset memory pressure comparison table.

[0009] Optionally, the method for obtaining the task queue length load rate weight value includes: Get the length of the run queue and the number of CPU cores on the node; The task queue length load rate is determined based on the run queue length and the number of CPU cores. The weight value of the task queue length load rate is determined based on the task queue length load rate and the preset task queue length load rate comparison table.

[0010] Optionally, the load balancing factor value = CPU utilization weight value * preset CPU weight + memory pressure weight value * preset memory pressure weight + task queue length load rate weight value * preset task queue length load rate weight.

[0011] Optionally, the communication frequency includes shared memory communication access frequency, pipe communication transmission frequency, socket communication connection frequency, socket communication data transmission sending frequency, socket communication data transmission receiving frequency, and message queue communication message frequency. By calculating the number of communication pairs between processes per unit time, the communication frequency of different communication types is distinguished, and an exponentially weighted moving average algorithm is used to process the communication frequency of each communication type to obtain a determined communication frequency value.

[0012] Optionally, the overall score of the target node = affinity weight value + load balancing factor value + topology weight.

[0013] Optionally, migration benefits include communication latency benefits and cache locality benefits. Communication latency benefits refer to the estimated reduction in communication latency between related processes after migration, while cache locality benefits refer to the estimated improvement in cache hit rate between related processes after migration.

[0014] In summary, this application includes at least one of the following beneficial technical effects: This invention optimizes everything from IPC communication monitoring to migration decisions, enabling automatic identification and optimization of process placement in complex multi-core NUMA systems, and significantly improving overall system performance. Attached Figure Description

[0015] Figure 1 This is a flowchart of a NUMA load balancing method based on inter-process communication awareness according to an embodiment of this application.

[0016] Figure 2This is a flowchart of step S300 of a NUMA load balancing method based on inter-process communication awareness according to an embodiment of this application.

[0017] Figure 3 This is a flowchart of step S500 of a NUMA load balancing method based on inter-process communication awareness according to an embodiment of this application.

[0018] Figure 4 This is a flowchart of step S501 of a NUMA load balancing method based on inter-process communication awareness according to an embodiment of this application.

[0019] Figure 5 This is a flowchart of step S502 of a NUMA load balancing method based on inter-process communication awareness according to an embodiment of this application.

[0020] Figure 6 This is a flowchart of step S503 of a NUMA load balancing method based on inter-process communication awareness according to an embodiment of this application. Detailed Implementation

[0021] The present application will be further described in detail below with reference to the accompanying drawings.

[0022] This specific embodiment is merely an explanation of this application and is not intended to limit it. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] In modern multi-core server systems, NUMA architecture has become the mainstream. However, traditional automatic NUMA balancing mechanisms optimize process placement and memory location based solely on memory access patterns, neglecting the critical impact of inter-process communication (IPC) on performance. When frequently communicating processes are distributed across different NUMA nodes, significant cross-node communication overhead occurs, leading to a substantial degrade in system performance.

[0025] Therefore, this invention introduces an IPC communication awareness mechanism based on existing NUMA load balancing. First, it monitors various IPC communication channels in real time, collects communication-related data, and updates IPC statistics. Then, it establishes an inter-process communication relationship graph based on the IPC statistics and calculates the relationship strength based on communication frequency and latency. Finally, it makes intelligent decisions to place relevant processes collaboratively on the optimal NUMA node.

[0026] The following are explanations of relevant technical terms: NUMA (Non-Uniform Memory Access) is a non-uniform memory access architecture where, in a multiprocessor system, processors access local memory faster than they access remote memory.

[0027] IPC (Inter-Process Communication) is a mechanism for transmitting data or signals between different processes, including shared memory, pipes, message queues, sockets, etc.

[0028] NUMA groups are data structures in the Linux kernel used to group processes with similar memory access patterns in order to coordinate NUMA load balancing decisions.

[0029] Memory affinity is the degree of association between a process and the physical memory location it accesses. High affinity means that the process primarily accesses the memory of its local node.

[0030] Figure 1 This is a flowchart of the NUMA load balancing method based on inter-process communication awareness of the present invention.

[0031] Step S100: Obtain communication data from the IPC.

[0032] Specifically, a comprehensive IPC communication monitoring mechanism is established in the kernel space. This involves injecting monitoring points at the entry points of various IPC system calls using kernel hook functions to collect data. Specifically, each time an IPC operation occurs, the IPC communication monitoring mechanism records the corresponding communication data for each type of IPC communication. This communication data is temporarily stored in a process-specific buffer and periodically aggregated. It's important to note that to avoid performance impact, a sampling and batch processing mechanism is used, performing detailed monitoring only on the communication paths of the four types of IPC communication.

[0033] IPC communication includes shared memory communication, pipe communication, socket communication, and message queue communication.

[0034] The communication data for shared memory communication includes the shared memory segment identifier, the accessing process PID, the access timestamp, and the size of the accessed data (in bytes). Monitoring dimension data is correlated with performance data; access frequency refers to the number of times the same shared memory segment is accessed per unit time, and data size refers to the amount of data accessed in a single access.

[0035] The communication data in piped communication includes the pipe file descriptor, the PID of the reading / writing process, the transmission timestamp, the size of the transmitted data, and the buffer usage. Monitoring dimensions are related to performance data: transmission frequency (the number of read / write operations per unit time through the same pipe), data throughput (the size of the transmitted data divided by the transmission time), transmission latency (the time difference between data submission at the writing end and data retrieval at the reading end), and buffer efficiency (the actual amount of transmitted data divided by the buffer size), reflecting the efficiency of pipe usage.

[0036] Socket communication data includes socket type (TCP / UDP / Unix Domain), sending / receiving process PID, port number / path, connection establishment time, sending / receiving timestamp, and packet size. Monitoring dimensions are correlated with performance data, including connection frequency (the number of new connections per unit time, reflecting inter-process communication activity), data transfer rate (the total amount of data sent and received divided by the connection duration), and round-trip latency (the request-response complete cycle time, or RTT).

[0037] Message queue communication data includes the message queue identifier, message type, sending / receiving process PID, operation timestamp, and message size. Monitoring dimensional data is correlated with performance data: message frequency (the number of messages sent / received per unit time), message latency (the time interval between message sending and receiving), and throughput (message processing rate per second).

[0038] Step S200: Determine the communication frequency, communication delay duration, and data transmission amount between each process pair based on the communication data.

[0039] It is important to know that the two processes involved in the process pair belong to different nodes.

[0040] Step S201: Analyze the communication frequency.

[0041] Communication frequencies include shared memory communication access frequency, pipe communication transmission frequency, socket communication connection frequency, socket communication data transmission sending frequency, socket communication data transmission receiving frequency, and message queue communication message frequency. The number of communications between process pairs (P1, P2) per unit time is calculated, and the communication frequencies of different communication types are distinguished. An exponentially weighted moving average algorithm is used to process the communication frequency of each communication type, giving higher weight to recent communication behaviors. A frequency threshold system is established to divide the communication frequencies into four levels: low frequency, medium frequency, high frequency, and ultra-high frequency.

[0042] It's important to know that Exponential Weighted Moving Average (EWMA) is a time series smoothing algorithm. Its core idea is to give higher weights to recent data and exponentially decrease the weights of older data; it doesn't need to store all historical data, only maintain a single state value; it is sensitive to new data while maintaining a certain level of stability.

[0043] Step S202: Analyze the communication delay duration.

[0044] Step S2021: Communication delay duration includes request-response mode communication delay duration and streaming communication delay duration.

[0045] Methods for calculating the latency of request-response communication include: Data acquisition at the sending end: Obtain the first precise timestamp of the communication request initiated by the sending process, which is recorded in the metadata header of the request message; Obtain the unique identifier of the communication request and the identifier of the receiving process for data matching during subsequent delay calculations; Obtain the NUMA node number where the sending process is currently located, and use it as the source node information for delay-distance analysis.

[0046] Data acquisition at the receiving end: Obtain the second precise timestamp when the receiving process begins processing the communication request, which is recorded when the request message arrives in the receive buffer; Obtain the third precise timestamp when the receiving process completes request processing and generates a response. This timestamp is recorded when the response message leaves the processing logic. Obtain the NUMA node number where the receiving process is currently located, and use it as the target node information for delay-distance analysis.

[0047] Response confirmation data acquisition: Get the fourth precise timestamp when the sending process receives the complete response, which is recorded when the response message is fully received and verified. Obtain the integrity verification result of the response message to ensure that the delay calculation is based on valid communication; It acquires retransmission or error recovery information that may occur during communication, and is used to distinguish between normal delays and abnormal delays.

[0048] Full cycle delay calculation: Based on the first and fourth precise timestamps, the duration of the complete communication cycle from initiating a communication request to receiving a complete response is calculated. This duration reflects the actual end-to-end performance of the communication.

[0049] Network transmission delay calculation: Based on the first and second precise timestamps, the pure data transmission duration is calculated, which reflects the basic transmission performance of the network IPC channel.

[0050] Processing delay calculation: Based on the second and third precise timestamps, calculate the service processing time at the receiving end. This time reflects the computing power and current load status of the receiving process.

[0051] Effective Delay Filtering: By analyzing the delay data of multiple communications, abnormal delay values ​​caused by system jitter, resource contention or temporary failures are excluded, and delay samples that reflect the true communication performance are retained. The delay samples are the delay duration of request-response mode communication.

[0052] Step S2022: The method for calculating the latency of streaming communication includes: Data block tag information acquisition: Obtain the sequence number, sending timestamp, and receiving timestamp for each data block. These markers are embedded by the sender when the data is generated.

[0053] Average transmission delay calculation: Based on the sending and receiving timestamps of multiple data blocks in the data stream, the arithmetic mean of the transmission delay of all data blocks is calculated. This value reflects the typical performance of the communication channel.

[0054] Delay fluctuation analysis: The standard deviation of the delay value of each data block is calculated based on the sending and receiving timestamps of multiple data blocks in the data stream. This quantifies the stability and predictability of the delay. High fluctuation indicates unstable communication quality.

[0055] Tail latency calculation: Filter the latency values ​​of the slowest 1% or 5% of data blocks in the latency distribution, as these tail latencies have the greatest impact on system stability.

[0056] Latency trend analysis: By performing moving averages and trend fitting on latency data over continuous time windows, long-term latency trends can be identified, and future performance changes can be predicted.

[0057] Step 203: Analyze the amount of data transmitted.

[0058] The total amount of data transferred between processes is accumulated, i.e., the data volume. It's important to know that the data volume is obtained using two transmission modes: large data transfers and small message communications.

[0059] Step S300: Determine the communication density index based on the communication frequency, communication delay duration, and data transmission volume, and classify the communication relationship level between each process pair based on the communication density index.

[0060] Calculation framework for communication density index: Input data (dimensions): Communication frequency (Hz), average latency (milliseconds), data transmission volume (bytes / second); Output: A series of values ​​from 0 to 100.

[0061] like Figure 2 As shown, the specific steps include S301 to S305.

[0062] Step S301: Determine the communication frequency score based on the communication frequency and the preset frequency score table.

[0063] If the communication frequency is ≤10Hz, then the communication frequency score = (communication frequency / 10) * 100; If 10Hz < communication frequency ≤ 100Hz, then the communication frequency score = (0.1 + 0.3 * log10(communication frequency / 10)) * 100; If 100Hz < communication frequency ≤ 1000Hz, then the communication frequency score = (0.4 + 0.4 * log10(communication frequency / 100)) * 100; If the communication frequency is greater than 1000Hz, then the communication frequency score is calculated as follows: {0.8 + 0.2 * min(1, (communication frequency - 1000) / 9000)} * 100.

[0064] It is important to know that the threshold range for communication frequency score is 0-100.

[0065] Step S302: Determine the communication delay score based on the communication delay duration and the delay duration score table.

[0066] The calculation process averages the delay of multiple communication delay durations over a certain period of time, i.e., the arithmetic mean mentioned above; The communication delay score is determined based on the average delay and the delay duration score table. If the average delay is ≤1ms, then the delay penalty = 0; If 1ms < average delay ≤ 5ms, then the delay penalty = (average delay - 1) * 10; If 5ms < average delay ≤ 10ms, then the delay penalty = 40 + (average delay - 5) * 8; If the average latency is greater than 10ms, then the latency penalty = 80 + min(20, (average latency - 10) * 2); Communication latency score = 100 - latency penalty. It's important to know that the threshold range for the communication latency score is 0-100.

[0067] Step 303: Determine the data transmission score based on the data transmission volume and the preset data transmission score table.

[0068] The transmitted data volume is normalized, i.e., normalized transmitted data volume = log10(transmitted data volume + 1) / log10(maximum expected transmitted data volume + 1). It should be noted that the maximum expected transmitted data volume is set according to the system memory bandwidth (e.g., 10GB / s).

[0069] The transmission mode is determined based on the normalized transmission data volume and the preset data volume score table, which stores the correspondence between the normalized transmission data volume and the transmission mode.

[0070] If the normalized data size is greater than 1KB, then the transmission mode is large block data transmission; If the normalized transmission data size is ≤1KB, then the transmission mode is small message communication.

[0071] The mode factor is determined based on the transmission mode and the preset data volume score table, which stores the correspondence between the transmission mode and the mode factor.

[0072] If the transmission mode is bulk data transmission, the mode factor is 0.8; If the transmission mode is small message communication, then the mode factor is 1.2.

[0073] The transmission data volume score is determined based on the normalized transmission data volume and the mode factor.

[0074] Data transfer volume score = Normalized data transfer volume × Pattern factor × 100. It's important to know that the threshold range for the data transfer volume score is 0-100.

[0075] Step S304: Determine the communication density index based on the communication frequency score, communication delay duration score, data transmission volume score, and preset weight table.

[0076] The preset weight table stores the weight values ​​for communication frequency, communication delay duration, and data transmission volume.

[0077] Communication density index = Communication frequency score * Communication frequency weight value + Communication delay duration score * Communication delay duration weight value + Data transmission volume score * Data transmission volume weight value.

[0078] In this invention, the communication frequency weight is 0.45, the communication delay weight is 0.40, and the data transmission volume weight is 0.15. That is, the communication density index = communication frequency * 0.45 + communication delay * 0.40 + data transmission volume * 0.15.

[0079] Step 305: Determine the communication relationship level based on the communication frequency, average delay, communication density index, and preset relationship level comparison table.

[0080] The preset relationship level comparison table stores the correspondence between communication frequency, average delay, communication density index and communication relationship level.

[0081] If the communication frequency is greater than 1 kHz, the average delay is less than 1 ms, and the communication density index is greater than 85, it indicates that there is close real-time cooperation between processes, and the communication relationship level is critical. If 100Hz < communication frequency ≤ 1kHz, 1ms < average delay ≤ 5ms, and 70 ≤ communication density index < 85, it indicates that there is frequent data exchange between processes, and the communication relationship level is a strong relationship. If 10Hz < communication frequency ≤ 100Hz, 5ms < average delay ≤ 10ms, and 40 ≤ communication density index < 70, it indicates that there is periodic interaction between processes, and the communication relationship level is medium. If the communication frequency is ≤10Hz, the average delay is >10ms, and the communication density index is <40, it indicates that there is only occasional communication need between processes, and the communication relationship level is weak.

[0082] It is important to know that when there is overlap in the data of communication frequency, average delay, and communication density indicators, the communication level is considered to be intermediate.

[0083] For example, if the communication frequency is greater than 1kHz, the average delay is less than 1ms and less than 5ms, and the communication density index is less than 70, then the communication level is medium.

[0084] Step S400: Determine the affinity weight value of each process pair to its node pair according to the communication relationship level between each process pair and the preset weight comparison table.

[0085] The preset weights lookup table is shown in Table 1:

[0086] Table 1 Preset Weight Comparison Table

[0087] Step S500: Obtain the load balancing factor of each node, and obtain the load balancing factor weight value of each node according to the load balancing factor and the preset weight correspondence, and then determine the load balancing factor value of each node based on the load balancing factor weight value.

[0088] like Figure 3 , Figure 4 , Figure 5 , Figure 6 As shown, it includes the following steps S501 to S504.

[0089] Step S501: Obtain the CPU utilization weight value, including: Step S5011: Obtain the running time and idle time of all CPU cores in each node; Step S5012: Calculate the sum of the running time and idle time of all CPU cores to obtain the total running time and total idle time; Step S5013: Determine CPU utilization based on total runtime and idle time, i.e.: CPU utilization = ((total running time - total idle time) / total running time) * 100%.

[0090] Step S5014: Determine the CPU utilization weight value based on the CPU utilization and the preset CPU utilization comparison table, i.e.; If the CPU utilization is ≤40%, then the CPU utilization weight value = 1.0; If 40% < CPU utilization ≤ 60%, then the CPU utilization weight value = 1.0 - (CPU utilization - 40%) × 0.005; If 60% < CPU utilization ≤ 80%, then the CPU utilization weight value = 0.9 - (CPU utilization - 60%) × 0.01; If 80% < CPU utilization ≤ 95%, then the CPU utilization weight value = 0.7 - (CPU utilization - 80%) × 0.02; If the CPU utilization rate is greater than 95%, then the CPU utilization rate weight value is 0.1.

[0091] The CPU utilization tier table is shown in Table 2: Table 2 CPU Utilization Tier Table

[0092] Step S502: Obtain the memory pressure weight value, including: Step S5021: Obtain the used memory and total memory of the node; Step S5022: Determine memory utilization based on used memory and total memory; Memory utilization = (used memory / total memory) * 100%.

[0093] Step S5023: Obtain the system's page swapping frequency, i.e., the number of times a page is swapped in and out per unit time; Step S5024: Determine memory pressure based on memory utilization, page swapping frequency, preset memory utilization weight value, and preset page swapping frequency weight value, i.e.: Memory pressure = memory utilization rate * preset memory utilization rate weight value + page swapping frequency * page swapping frequency weight value.

[0094] It's important to know that the preset memory utilization weight value + preset page swapping frequency weight value = 1.

[0095] Step S5025: Determine the memory pressure weight value based on the memory pressure and the preset memory pressure comparison table, that is: If the memory pressure is ≤0.3, then the memory pressure weight value = 1.0; If 0.3 < memory pressure ≤ 0.5, then the memory pressure weight value = 1.0 - (memory pressure - 0.3) * 0.25; If 0.5 < memory pressure ≤ 0.7, then the memory pressure weight value = 0.95 - (memory pressure - 0.5) * 0.75; If 0.7 < memory pressure ≤ 0.9, then the memory pressure weight value = 0.85 - (memory pressure - 0.7) * 1.75; If the memory pressure is greater than 0.9, then the memory pressure weight value is 0.2.

[0096] The memory stress grading table is shown in Table 3: Table 3 Memory Stress Level Table

[0097] Step S503: Obtain the task queue length load rate weight value, including: Step S5031: Obtain the run queue length and CPU core count on the node. The run queue length is the number of tasks that are in a runnable state on the node, and the CPU core count is the total number of CPU cores on the node.

[0098] Step S5032: Determine the task queue length load rate based on the run queue length and the number of CPU cores, i.e., task queue length load rate = run queue length / number of CPU cores.

[0099] Step S5033: Determine the weight value of the task queue length load rate based on the task queue length load rate and the preset task queue length load rate comparison table, that is: If the task queue length load rate is ≤ 1.0, then the task queue length load rate weight value = 1.0. If 1.0 < Task queue length load rate ≤ 2.0, then the task queue length load rate weight value = 1.0 - (Task queue length load rate - 1.0) × 0.1 If 2.0 < Task queue length load rate ≤ 3.0, then the task queue length load rate weight value = 0.9 - (Task queue length load rate - 2.0) × 0.2 If 3.0 < Task queue length load rate ≤ 4.0, then the task queue length load rate weight value = 0.7 - (Task queue length load rate - 3.0) × 0.3 If the task queue length load rate is greater than 4.0, then the task queue length load rate weight value is 0.1.

[0100] The task queue length load rate table is shown in Table 4: Table 4 Task Queue Length Load Rate Table

[0101] Step S504: Determine the load balancing factor value based on the CPU utilization weight value, memory pressure weight value, task queue length load rate weight value, preset CPU weight, preset memory pressure weight, and preset task queue length load rate weight.

[0102] Load balancing factor value = CPU utilization weight value * preset CPU weight + memory pressure weight value * preset memory pressure weight + task queue length load rate weight value * preset task queue length load rate weight.

[0103] It's important to know that the preset CPU weight + memory pressure weight + preset task queue length load rate weight = 1.

[0104] Step S600: For any given process, determine the node where it is located as the source node and the other nodes as target nodes. Based on the distance between the target node and the source node, determine the topology factor weight of each target node according to the preset topology factor comparison table.

[0105] The topological factor comparison table is shown in Table 5: Table 5 Topological Factor Comparison Table

[0106] This invention considers the physical topology of NUMA nodes, including interconnection distances and bandwidth between nodes. For related processes that cannot be placed on the same node, node pairs with closer interconnection distances are preferred. In multi-layer NUMA topologies, cache coherency and the distribution of memory controllers are taken into account.

[0107] Step S700: Determine the comprehensive score of each target node based on the affinity weight and topology weight of the target node relative to the source node, as well as the load balancing factor value of the target node itself, and select the node with the highest comprehensive score as the optimal source node.

[0108] Overall score = affinity weight value + load balancing factor value + topology weight.

[0109] After calculating the overall score of all nodes, sort the overall scores of all nodes in descending or ascending order, and then select the node with the highest overall score as the optimal node.

[0110] Step S800: Obtain the migration benefits and migration costs of the optimal node.

[0111] Migration benefits include communication latency benefits and cache locality benefits. Communication latency benefits refer to the estimated reduction in communication latency between related processes after migration, while cache locality benefits refer to the estimated improvement in cache hit rate between related processes after migration.

[0112] The methods for calculating communication delay benefits include: Obtain the average communication latency L_before (nanoseconds) between the process pairs corresponding to the target node and the source node before migration, the estimated communication latency L_after (nanoseconds) after migration, and the number of communications per unit time F (times / second). The communication latency gain per second is determined based on the average communication latency L_before (nanoseconds) between the process pairs corresponding to the target node and the source node before migration, the estimated communication latency L_after (nanoseconds) after migration, and the number of communications per unit time F (times / second). Single communication latency reduction = L_before - L_after (nanoseconds); The benefit of communication delay per second = F*(L_before-L_after) (nanoseconds / second).

[0113] The communication delay benefit is determined based on the communication delay benefit per second and a first preset benefit time window, i.e.: Communication delay benefit = Communication delay benefit per second * First preset benefit time window. It's important to know that the first preset benefit time window can be 1 second, 5 seconds, or 10 seconds.

[0114] Methods for calculating cache locality benefits include: Get the cache miss rate M_before (percentage) between the process pairs corresponding to the target node and the source node before migration, the estimated cache miss rate M_after (percentage) after migration, the number of memory accesses A (times / second) and the cache miss penalty time P (nanoseconds / time). The per-second benefit of mixed storage is determined based on the cache miss rate M_before (percentage) between the processes corresponding to the target node and the source node before migration, the estimated cache miss rate M_after (percentage) after migration, the number of memory accesses A (times / second), and the cache miss penalty time P (nanoseconds / time). Access miss rate reduction = M_before - M_after (percentage); The number of misses reduced per second = A * ((M_before-M_after) / 100) (misses / second); The benefit of caching time per second = the reduction in misses per second - P (nanoseconds / second).

[0115] The cache locality benefit is determined based on the benefit of cache time per second and a second preset benefit time window, i.e.: Cache locality benefit = Cache time per second benefit * Second preset benefit time window. It's important to know that the first preset benefit time window is 1 second, 5 seconds, or 10 seconds.

[0116] In summary, migration benefits are determined based on communication delay benefits, cache locality benefits, preset communication delay benefit weight values, and preset cache locality benefit weight values.

[0117] Migration benefit = Communication delay benefit * Preset communication delay benefit weight value + Cache locality benefit * Preset cache locality benefit weight value.

[0118] It is important to know that the preset communication delay benefit weight value + preset cache locality benefit weight value = 1.

[0119] Migration costs include direct costs such as process state saving and restoration, TLB refresh, and cache invalidation; memory migration costs: if the process's resident memory pages need to be migrated, the overhead of page migration is calculated; system disturbance costs: the temporary impact of migration operations on the overall system performance; topology distance costs: the additional overhead caused by the physical distance between the source node and the target node is considered.

[0120] Step S900: Determine whether to migrate the process on the source node to the optimal node based on the migration benefits and migration costs.

[0121] If the migration benefit is greater than or equal to 2 * migration cost, the target process will be migrated to the optimal node at an appropriate scheduling time (the next clock interrupt or load balancing cycle). If the migration benefit is less than 2 * migration cost, then the target process will not be migrated to the optimal node.

[0122] The system executes specific migration operations through the kernel scheduler and continuously monitors the optimization effects. Through the scheduler's task migration interface, the target process is migrated from the source CPU to the CPU of the optimal node. For already bound memory pages, a memory migration mechanism moves them to the optimal node, updating the process's CPU affinity and memory policy to ensure subsequent allocation to the optimal node. After migration, all relevant kernel data structures are updated synchronously.

[0123] This invention optimizes everything from IPC communication monitoring to migration decisions, enabling automatic identification and optimization of process placement in complex multi-core NUMA systems, and significantly improving overall system performance.

[0124] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A NUMA load balancing method based on inter-process communication awareness, characterized in that, include: Obtain IPC communication data; The communication frequency, communication delay duration, and amount of data transmitted between each process pair are determined based on the communication data. The communication density index is determined based on the communication frequency, communication delay duration, and data transmission volume, and the communication relationship level between each process pair is classified based on the communication density index. The affinity weight value of each process pair to its node pair is determined based on the communication relationship level between each process pair and the preset weight comparison table. Obtain the load balancing factor of each node, and obtain the load balancing factor weight value of each node according to the load balancing factor and the preset comparison relationship. Then determine the load balancing factor value of each node based on the load balancing factor weight value. For any given process, the node where it is located is determined as the source node, and the other nodes are determined as target nodes. Based on the distance between the target node and the source node, the topological weight of each target node is determined according to a preset topological factor lookup table. The comprehensive score of each target node is determined based on the affinity weight and topology weight of the target node relative to the source node, as well as the load balancing factor value of the target node itself, and the target node with the highest comprehensive score is selected as the optimal node of the source node. To determine the optimal node's migration benefits and costs; Determine whether to migrate processes from the source node to the optimal node based on migration benefits and migration costs. If the migration benefit is greater than or equal to 2 * migration cost, then the process on the source node will be migrated to the optimal node. If the migration benefit is less than 2 * migration cost, then the process on the source node will not be migrated to the optimal node.

2. The NUMA load balancing method based on inter-process communication awareness according to claim 1, characterized in that, The method of classifying communication relationship levels based on communication frequency, communication delay duration, and transmitted data volume includes: The communication frequency score is determined based on the communication frequency and the preset frequency score table; The communication delay score is determined based on the communication delay duration and the preset delay duration score table. The score for the transmitted data volume is determined based on the transmitted data volume and the preset data volume score table. The communication density index is determined based on the communication frequency score, communication delay duration score, data transmission volume score, and a preset weight table. The communication relationship level is determined based on the communication frequency, average delay duration, communication density index, and a preset relationship level comparison table.

3. The NUMA load balancing method based on inter-process communication awareness according to claim 1, characterized in that, The load balancing factors for each node include CPU utilization, memory pressure, and task queue length load rate. The weight values ​​of the load balancing factors include CPU utilization weight value, memory pressure weight value, and task queue length load rate weight value.

4. The NUMA load balancing method based on inter-process communication awareness according to claim 3, characterized in that, Methods for obtaining CPU utilization weight values ​​include: Obtain the runtime and idle time of all CPU cores within each node; Calculate the sum of the running time and idle time of all CPU cores to obtain the total running time and total idle time; CPU utilization is determined based on the total running time and total idle time. The CPU utilization weight value is determined based on the CPU utilization and the preset CPU utilization comparison table.

5. A NUMA load balancing method based on inter-process communication awareness according to claim 3, characterized in that, Methods for obtaining memory pressure weight values ​​include: Get the used memory and total memory of the node; Memory utilization is determined based on the used memory and total memory. Obtain the system's page swapping frequency, i.e., the number of times a page is swapped in and out per unit of time; Memory pressure is determined based on memory utilization, page swapping frequency, preset memory utilization weight value, and preset page swapping frequency weight value. The memory pressure weight value is determined based on the memory pressure and the preset memory pressure comparison table.

6. The NUMA load balancing method based on inter-process communication awareness according to claim 3, characterized in that, The method for obtaining the task queue length load rate weight value includes: Get the length of the run queue and the number of CPU cores on the node; The task queue length load rate is determined based on the run queue length and the number of CPU cores. The weight value of the task queue length load rate is determined based on the task queue length load rate and the preset task queue length load rate comparison table.

7. A NUMA load balancing method based on inter-process communication awareness according to claim 3, characterized in that, Load balancing factor value = CPU utilization weight value * preset CPU weight + memory pressure weight value * preset memory pressure weight + task queue length load rate weight value * preset task queue length load rate weight.

8. A NUMA load balancing method based on inter-process communication awareness according to claim 1, characterized in that, The communication frequencies include shared memory communication access frequency, pipe communication transmission frequency, socket communication connection frequency, socket communication data transmission sending frequency, socket communication data transmission receiving frequency, and message queue communication message frequency. By calculating the number of communications between process pairs per unit time, the communication frequencies of different communication types are distinguished. An exponentially weighted moving average algorithm is used to process the communication frequency of each communication type to obtain a determined communication frequency value.

9. A NUMA load balancing method based on inter-process communication awareness according to claim 1, characterized in that, The overall score of the target node = affinity weight value + load balancing factor value + topology weight.

10. A NUMA load balancing method based on inter-process communication awareness according to claim 1, characterized in that, Migration benefits include communication latency benefits and cache locality benefits. Communication latency benefits refer to the estimated reduction in communication latency between related processes after migration, while cache locality benefits refer to the estimated improvement in cache hit rate between related processes after migration.

Images