Pipeline communication optimization method and system under NUMA architecture

Abstract

The invention discloses a method and a system for optimizing pipeline communication under an NUMA (Non Uniform Memory Access) architecture, and the method comprises the following steps: when a pipeline is created, distributing a pipeline communication memory on a specified node; detecting the communication data rate of each pipeline; the threshold value of each pipeline is calculated according to the type and priority of the process, the communication data rate of each pipeline is compared with the corresponding threshold value, and the important process or the process of which the communication data rate of the corresponding pipeline is greater than the threshold value is migrated to a specified node to run; and keeping the non-important process or the process of which the communication data rate of the corresponding pipeline is smaller than the threshold value at the corresponding original node for operation. According to the method, the NUMA hardware architecture characteristics can be fully played, so that the system operation efficiency is improved, and the performance problem caused by cross-node memory access of NUMA architecture server pipeline communication is solved.

Description

Technical Field

[0001] The present invention relates to computer technology, and in particular to a pipeline communication optimization method and system under a NUMA architecture. Background Art

[0002] The NUMA architecture is suitable for large-scale multi-processor systems, especially when the performance improvement of a single processor is limited by physical limits. The overall computing power of the hardware can be improved by increasing the number of processors. At present, domestic mainstream CPU manufacturers basically design server CPUs based on the NUMA architecture, such as Feiteng S5000C, Huawei HiSilicon Kunpeng 920, and Haiguang 4, which are all NUMA architectures. Because the memory access time of the NUMA architecture depends on the relative position between the processor and the memory, the processor accesses the local memory faster than the non-local memory, so the software design needs to consider the characteristics of NUMA, give full play to the hardware characteristics, and optimize the performance of the system.

[0003] Pipeline communication is an important inter-process communication method in Linux system. It allows different processes to share data and transmit messages. It is one of the important mechanisms to achieve inter-process synchronization and communication. Its communication efficiency will directly affect the running efficiency of the process. However, the current design and implementation of pipeline communication in Linux kernel does not fully consider the characteristics of NUMA hardware architecture. For example, when important processes communicate across nodes through pipelines, the communication bandwidth will be limited by the communication bandwidth across nodes. At the same time, the pipeline communication delay will also increase, which will affect the running efficiency of important processes.

[0004] The Chinese invention patent application "A pipeline communication method and device" (application number CN201310034616.6) discloses a pipeline communication method and device, including establishing a data storage space, storing data in the data storage space, assigning characteristic parameters corresponding to the data stored in the data storage space, reading corresponding data from the data storage space according to the characteristic parameters, and using a common space to connect with the input and output pipelines to uniformly manage and store the amount of data input by the input pipeline. The output pipeline of any process in the system can read the required amount of data from the common space, thereby greatly reducing the scale and complexity of the pipeline communication system and improving the reliability of the system. This solution mainly achieves the purpose of improving system reliability by reducing the scale and complexity of the pipeline communication system, and cannot solve the performance problem caused by cross-node memory access during pipeline communication under NUMA architecture servers.

[0005] Chinese invention patent application "Memory Access Method, Device and System" (application number CN201310257057.5) discloses a memory access method, device and system, wherein the memory access method includes a node controller receiving monitoring information sent by an operating system, the monitoring information carrying information of a monitored memory in a first node to which the node controller belongs, the monitored memory being memory resources occupied by a target process on the first node, the target process being a process running on the central processing unit CPU of the first node and accessing the memory of an accessed node other than the first node in a server system; if it is monitored that the frequency of accesses to the memory of the accessed node by the target process occupying the monitored memory is greater than or equal to a threshold, the information of the accessed node is sent to the operating system to migrate the target process to the accessed node according to the information of the accessed node; the remote memory access is converted into a local memory access or a neighboring memory access, thereby reducing the time for the target process to access the memory and effectively improving the performance of the server system. This solution is mainly implemented through the node controller NC chip hardware. The operating system sends the monitoring information to the NC chip hardware, which is then monitored by the NC chip hardware. When the NC chip hardware monitors that the frequency of remote memory access is too high, it sends information to the operating system to migrate the process. The implementation of the solution is highly dependent on the hardware node controller NC chip. It is not a universal implementation solution for the operating system itself and cannot solve the performance problems caused by cross-node memory access during operating system pipeline communication. Summary of the invention

[0006] The technical problem to be solved by the present invention is as follows: In view of the above-mentioned problems in the prior art, a pipeline communication optimization method and system under a NUMA architecture are provided, which can give full play to the characteristics of the NUMA hardware architecture, thereby improving the system operation efficiency and solving the performance problems caused by cross-node memory access of pipeline communication in NUMA architecture servers.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is: A pipeline communication optimization method under a NUMA architecture comprises the following steps: When creating a pipeline, allocate pipeline communication memory on the specified node; Detect the communication data rate of each pipeline; The threshold of each pipeline is calculated according to the type and priority of the process, and the communication data rate of each pipeline is compared with the corresponding threshold. Important processes or processes with a communication data rate greater than the threshold are migrated to the specified node for execution, while non-important processes or processes with a communication data rate less than the threshold are retained on the corresponding original node for execution.

[0008] Furthermore, the specified node is specifically a node specified by a user, or a node running by a process for creating a pipeline.

[0009] Furthermore, when detecting the communication data rate of each pipeline, it includes: Get the historical communication data rate of the current pipeline, and wait for the process corresponding to the current pipeline to write data to the current pipeline or read data from the current pipeline; When a process writes data to or reads data from the current pipe, the read data or write data in the current time period in the current pipe is accumulated and recorded to obtain the communication data volume in the current time period; The communication data volume of the current time period is divided by the duration of the current time period to obtain the current communication data rate of the current pipeline; The current communication data rate is smoothed using the historical communication data rate and a preset amplification factor.

[0010] Furthermore, when the current communication data rate is smoothed using the historical communication data rate and the preset amplification factor, the expression is as follows: S = Cur_S * GF + Pre_S * (1 - GF) Where S represents the communication data rate after smoothing, Cur_S represents the current communication data rate, Pre_S represents the historical communication data rate, and GF represents the amplification factor, which ranges from (0, 1].

[0011] Further, after waiting for the process corresponding to the current pipe to write data to the current pipe or read data from the current pipe, it also includes: The decay time is determined according to the priority of the process corresponding to the current pipeline. If the waiting time exceeds the decay time, the historical communication data rate is decayed using the preset decay factor to obtain the current communication data rate, and then the process corresponding to the current pipeline continues to write data to the current pipeline or read data from the current pipeline.

[0012] Furthermore, when the preset attenuation factor is used to attenuate the historical communication data rate, the expression is as follows: S = Pre_S *AF Where S represents the communication data rate after attenuation, Pre_S represents the historical communication data rate, and AF represents the attenuation factor, which ranges from (0, 1].

[0013] Furthermore, when calculating the threshold of each pipeline according to the type and priority of the process, it includes: Get the scheduling policy of the current process; If the scheduling policy of the current process is the real-time scheduling policy, the threshold of the pipeline corresponding to the current process is set to 0; If the scheduling policy of the current process is not a real-time scheduling policy, the priority of the current process is obtained, and the priority is substituted into the threshold calculation function to obtain the threshold of the pipeline corresponding to the current process.

[0014] Furthermore, the threshold calculation function is: pipe->threshold = PIPE_THRESHOLD * pow(a, task_nice(current)) Among them, PIPE_THRESHOLD represents the threshold when the process priority is 0, the pow(a, task_nice(current)) function implements the power operation logic of a, task_nice() is a function for obtaining the process priority, and current is the struct task_struct structure variable representing the current process.

[0015] Furthermore, before obtaining the scheduling policy of the current process, it also includes: If the current process is a designated important process, the threshold of the pipeline corresponding to the current process is set to 0; If the current process is a designated unimportant process, the threshold of the pipeline corresponding to the current process is set to the maximum value of the storage type.

[0016] The present invention also proposes a pipeline communication optimization system under a NUMA architecture, comprising a microprocessor and a computer-readable storage medium connected to each other, wherein the microprocessor is programmed or configured to execute any one of the pipeline communication optimization methods under a NUMA architecture.

[0017] Compared with the prior art, the advantages of the present invention are: The present invention aims at the characteristics of NUMA architecture hardware, realizes the detection of the communication bandwidth of each pipeline in the system by calculating the communication data rate of the pipeline, and distinguishes the importance of system processes at the same time. For processes with smaller pipeline communication bandwidth or non-important processes, they are allowed to communicate across node memory, and for processes with larger pipeline communication bandwidth or important processes, they are scheduled to communicate on the same node, so as to improve their pipeline communication bandwidth and reduce pipeline communication delay, thus giving full play to the hardware characteristics of NUMA architecture and improving system operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The following is a brief flowchart of an embodiment of the present invention.

[0019] Figure 2 The figure is a flow chart of calculating the communication data rate according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention is further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the present invention is not limited thereby.

[0021] Before introducing the specific embodiments of the present invention, related concepts or terms are first explained.

[0022] NUMA: NUMA (Non Uniform Memory Access) is a design for multi-processors, and its memory access time depends on the memory location of the processor. The NUMA architecture has the concept of node. Each node has its own internal CPU, bus, memory, and I / O slots. At the same time, each node's own internal CPU can also access the memory and I / O slots in other nodes through the interconnection bus. Therefore, each CPU can access the memory of the entire system. However, under NUMA, the processor accesses its own local memory (that is, the CPU and memory are located in the same node) much faster than accessing non-local memory (also called remote memory, that is, the CPU and memory are not located in the same node) in terms of speed and latency. This is also the origin of non-uniform memory access NUMA. Due to this feature, in order to better exert system performance, it is necessary to minimize the information interaction between different nodes when developing applications.

[0023] Pipe: Pipe is a Linux inter-process communication mechanism that allows one process to pass data to another process. There are two main types of pipes: anonymous pipes and named pipes. Anonymous pipes are usually used for communication between parent and child processes or sibling processes, while named pipes allow communication between unrelated processes.

[0024] Embodiment 1 A pipeline communication optimization method under NUMA architecture detects pipeline communication bandwidth, schedules processes with larger communication bandwidth or important processes to run on the same node, prevents cross-node memory communication access communication, and allows cross-node memory communication for non-important processes with smaller communication bandwidth, giving full play to the characteristics of NUMA hardware architecture, thereby improving system operation efficiency.

[0025] like Figure 1 As shown, the method of this embodiment includes the following steps: S1) declare and initialize the required data structures; S2) When creating a pipeline, allocate pipeline communication memory on the specified node; S3) detecting the communication data rate of each pipeline; S4) Calculate the threshold of each pipeline according to the type and priority of the process, compare the communication data rate of each pipeline with the corresponding threshold, migrate important processes or processes whose corresponding pipeline communication data rate is greater than the threshold to the specified node to avoid cross-node memory access and improve the efficiency of its pipeline communication, and keep non-important processes or processes whose corresponding pipeline communication data rate is less than the threshold on the corresponding original node to achieve cross-node memory communication.

[0026] The following is a detailed description of each step.

[0027] In this embodiment, step S1 is used to define and initialize the data and variables to be used in the subsequent steps, which specifically includes: S11) in the struct pipe_inode_info structure in the include / linux / pipe_fs_i.h file, define a variable int nid for specifying the node node allocated to the pipe communication memory, define a variable unsigned longtraffic_num for recording the amount of communication data of the pipe for a period of time, define a variable unsigned long pipe_time for calculating the time point of the pipe communication data rate, define a variable int pipe_speed for storing the calculated pipe communication data rate, define a variable int threshold for storing the threshold value calculated by the process priority, define a variable int decay_time for storing the decay time of the process pipe communication, define a variable struct timer_listspeed_timer for the pipe communication rate timer, define a pipe communication rate increment factor GF constant as 0.6, and define a pipe communication rate decay factor AF constant as 0.3; S12) After allocating the memory of the struct pipe_inode_info structure in the kernel function alloc_pipe_info(), the traffic_num and pipe_time in the structure are initialized to 0, and timer_setup(&pipe->speed_timer, speed_timer_function, 0) is called to initialize the timer function to speed_timer_function. The speed_timer_function function completes the attenuation of the pipeline communication data rate, and the attenuation factor is the constant AF.

[0028] In step S2 of this embodiment, when the process creates a pipeline, the memory used for pipeline communication is allocated to a specific node. In step S2, the specified node can be a node specified by the user. If the user does not specify, it can also default to the node on which the process that creates the pipeline runs.

[0029] In the default case, the specific implementation code is as follows: S21) Set the node that allocates the pipeline communication memory to the running node of the process that currently creates the pipeline: pipe->nid = cpu_to_node(task_cpu(current)); task_cpu(current) is used to obtain the CPU number of the process that currently creates the pipeline, and cpu_to_node() is used to obtain the node to which the specified CPU number belongs; S22) Allocate pipe communication memory on the node specified in step S21: pipe->bufs = kmalloc_node(pipe_bufs*sizeof(struct pipe_buffer), GFP_KERNEL_ACCOUNT|__GFP_ZERO, pipe->nid); where pipe->nid represents the specified node, pipe_bufs * sizeof(struct pipe_buffer) represents the allocated memory size, pipe_bufs is an integer representing the number of pipe buffers to be allocated, sizeof(struct pipe_buffer) is the size of each pipe buffer, and struct pipe_buffer is a structure used to represent the buffer in the pipe. GFP_KERNEL_ACCOUNT is a flag for kernel memory allocation, indicating that memory is allocated in kernel space, and the allocated memory is included in the memory usage statistics of the process; __GFP_ZERO indicates that the allocated memory will be cleared. That is, after the allocation is completed, the allocated memory area will be initialized to all zeros.

[0030] In the case specified by the user, only step S22 is executed, and memory is allocated according to the node specified by the user. The allocated memory will be cleared, and the allocated memory will be included in the memory usage statistics of the process.

[0031] Step S3 of this embodiment is to detect the communication data rate of each pipeline's communication link, using a passive measurement method to count the total amount of data for a period of time when the process writes data to the pipeline or reads data from the pipeline, and then divide the total amount of data written or read by the time value of counting the amount of data written or read to measure the communication data rate. The following steps are included: S31) obtaining the historical communication data rate of the current pipe, waiting for the process corresponding to the current pipe to write data to the current pipe or read data from the current pipe, determining the decay time according to the priority of the process corresponding to the current pipe, and if the waiting time exceeds the decay time, using a preset decay factor to decay the historical communication data rate to obtain the current communication data rate, and then continue to wait for the process corresponding to the current pipe to write data to the current pipe or read data from the current pipe; Specifically, when the pipeline has not communicated for a period of time, the measured pipeline communication data rate needs to be attenuated. Assuming that the historical communication data rate of the current pipeline, that is, the communication data rate measured last time is Pre_S, the attenuation factor is AF, and the AF value range is [0, 1), that is, AF is greater than or equal to 0 and less than 1. Assuming that the attenuation time is DT, when the pipeline does not detect communication data every DT time period, the preset attenuation factor is used to attenuate the historical communication data rate. The expression is as follows: S = Pre_S *AF; It can be seen from the calculation formula that the smaller the attenuation factor AF is, the faster the pipeline communication rate S decays. When the attenuation factor AF is 0, it means that when the pipeline has no communication for a period of time, the pipeline communication rate S directly decays to 0; In this embodiment, the decay time DT can be uniformly set to a default empirical value or determined by the process priority. When the decay time DT is determined by the process priority, the higher the process priority, the greater the decay time DT, and the lower the process priority, the smaller the decay time DT; S32) When the process writes data to the current pipe or reads data from the current pipe, the read data or write data in the current time period in the current pipe is accumulated and recorded to obtain the communication data volume in the current time period; S33) dividing the communication data volume of the current time period by the duration of the current time period to obtain the current communication data rate of the current pipeline, that is, S = N / T, where S is the pipeline communication data rate, and N is the amount of data written to or read from the pipeline within the time T, and then using the historical communication data rate and the preset increase factor to smooth the current communication data rate; Specifically, in order to prevent the very high pipeline communication data rate measured instantaneously from causing frequent scheduling migration of processes, an amplification factor GF is introduced in this embodiment to smooth the measured pipeline communication data rate. The value range of GF is (0, 1], that is, GF is greater than 0 and less than or equal to 1. Assuming that the historical pipeline communication data rate, that is, the communication data rate measured last time, is Pre_S, and the currently measured pipeline communication data rate is Cur_S, then when the historical communication data rate and the preset amplification factor are used to smooth the current communication data rate, the expression is as follows: S = Cur_S * GF + Pre_S * (1 - GF); It can be seen from the calculation formula that the smaller the amplification factor GF is, the smaller the contribution of the currently measured pipeline communication data rate to the pipeline communication data rate S is. When the pipeline communication data rate is increasing, the calculated pipeline communication data rate S increases in speed, and when the pipeline communication data rate is decreasing, the calculated pipeline communication data rate S decelerates in speed. In other words, the smaller the amplification factor GF is, the smoother the calculated pipeline communication rate is. When the amplification factor GF is 1, it means that no smoothing is performed.

[0032] The specific process of the above steps is as follows Figure 2 As shown, specifically including; 1) Get the current system time point: unsigned long cur_time = jiffies; 2) Reset the timer at 1 second intervals: mod_timer(&pipe->speed_timer, jiffies +msecs_to_jiffies(1000)), that is, if the pipe communication data is 0 within 1 second, it means that the waiting process has written data to the current pipe or read data from the current pipe for more than the decay time, and the pipe communication rate is decayed. The amplitude of each decay is AF, and the calculation method is pipe_speed = pipe_speed * AF, where pipe_speed is the pipe communication data rate; mod_timer() is a function implemented by the kernel to modify the timeout of an existing timer, and msecs_to_jiffies() is a macro implemented by the kernel to convert milliseconds (ms) to the kernel's timing unit jiffies; 3) When the system calls the kernel function pipe_write() to send data to the pipe or calls the kernel function pipe_read() to read data from the pipe, the amount of data sent or read is accumulated and recorded: pipe->traffic_num = pipe->traffic_num + total_len, where total_len is the amount of data sent or read, in bytes B; 4) Determine the difference between cur_time and pipe->pipe_time: int pass_time = jiffies_to_msecs(cur_time - pipe->pipe_time), if pass_time is greater than or equal to 1000, jump to step 5), otherwise jump to step 3); where jiffies_to_msecs() is a macro implemented by the kernel, used to convert jiffies (kernel timing unit) to milliseconds (ms); 5) Calculate the current pipe communication data rate: int pipe_speed = pipe->traffic_num / pass_time; 6) Smooth the pipe communication data rate: pipe->pipe_speed = pipe_speed * GF + pipe->pipe_speed * (1 - GF); 7) Reset the pipe communication data volume: pipe->traffic_num = 0; 8) Set pipe->pipe_time to the current system time point: pipe->pipe_time = cur_time.

[0033] In the above steps, step 1 and step 2 correspond to step S31, step 3 and step 4 correspond to step S32, and step 5 to step 8 correspond to step S33. By executing one round of the above steps, the communication data rate of a time period (1 second in this embodiment) of the pipeline is detected, and the detection result of this round can be used for threshold comparison in subsequent steps and as the historical communication data rate of the next round. It should be noted that if the statistical time period when detecting the communication data rate of each pipeline is greater than the time period of executing one round of the above steps, multiple rounds of the above steps can be executed, and the detection result of the last round is output for threshold comparison in subsequent steps.

[0034] In step S4 of this embodiment, by comparing the communication data rate of the pipeline communication link with the threshold, it is determined whether the process that creates the pipeline and uses the pipeline communication needs to be migrated to the same node as the pipeline communication memory. Therefore, when calculating the threshold of each pipeline according to the type and priority of the process, the threshold of the corresponding pipeline is calculated for all processes using the pipeline communication, including the following situations: If the current process is a designated important process, the threshold of the pipeline corresponding to the current process is set to 0. When the threshold is set to 0, it means that this is a critical communication pipeline, such as the communication pipeline of a real-time process. The communication efficiency is very important. Therefore, regardless of the pipeline communication rate, the communication process needs to be scheduled to run on the same node as the pipeline communication memory to improve the pipeline communication efficiency. If the current process is a designated unimportant process, the threshold of the pipeline corresponding to the current process is set to the maximum value of the storage type. When the threshold is set to the maximum value of the storage type, it means that this is an insignificant communication pipeline and the communication efficiency is not important. Therefore, no matter how high or low the pipeline communication rate is, there is no need to schedule the communication process to run on the same node as the pipeline communication memory. If the current process does not belong to the above two situations, the threshold of the corresponding pipeline can be set to a default empirical value or determined by the process priority. When the threshold is determined by the process priority, the higher the process priority, the lower the threshold, and the lower the process priority, the higher the threshold.

[0035] Specifically, step S4 can be implemented by the following steps: S41) define the constant PIPE_THRESHOLD as 100, which represents the pipe communication threshold of the process priority nice as 0, in units of B / ms, that is, when the pipe communication rate of the process priority nice as 0 is greater than 100 B / ms, the process needs to be migrated to the same node as the pipe communication memory; S42) Obtain the scheduling policy of the current process through current->policy to determine whether the current process is a real-time process. If the scheduling policy of the current process is SCHED_FIFO or SCHED_RR, the scheduling policy of the current process is a real-time scheduling policy. Set the threshold pipe->threshold of the pipe corresponding to the current process to 0 and jump to step S44. Otherwise, the current process is a normal process and execute step S43, where current is a structtask_struct structure variable defined by the kernel representing the current process, and its member policy sets the scheduling policy type. SCHED_FIFO and SCHED_RR are real-time scheduling policy constants defined by the kernel. S43) If the scheduling policy of the current process is not a real-time scheduling policy, the priority of the current process is obtained through task_nice(current), and the priority is substituted into the threshold calculation function to obtain the threshold of the pipeline corresponding to the current process. The threshold calculation function is: pipe->threshold = PIPE_THRESHOLD * pow(a, task_nice(current)) Among them, PIPE_THRESHOLD represents the threshold value when the process priority is 0, the pow(a, b) function implements the function of the b-power operation logic of a. In this embodiment, a is 1.25, b is task_nice(current), that is, the process priority, task_nice() is a function for obtaining the process priority, and current is a struct task_struct structure variable representing the current process; S44) Determine whether pipe->pipe_speed is greater than pipe->threshold, if so, execute step S45, otherwise terminate step S44 and exit to keep the current process running at its original node; S45) Schedule the current process to run at the node where the pipe communication memory is located, and execute the statement: sched_setaffinity(current->pid, cpumask_of_node(pipe->nid)); wherein sched_setaffinity() is a function implemented by the kernel to specify the cpu set on which the process runs, current->pid is the pid of the current process, and cpumask_of_node() is a function implemented by the kernel to obtain the CPU mask on the specified node.

[0036] Embodiment 2 The present invention also proposes a pipeline communication optimization system under a NUMA architecture, comprising a microprocessor and a computer-readable storage medium connected to each other, wherein the microprocessor is programmed or configured to execute any one of the pipeline communication optimization methods under a NUMA architecture.

[0037] In summary, the present invention discloses a pipeline communication optimization method and system under NUMA architecture. First, when the communication pipeline is created, the memory used for pipeline communication is allocated to a specific node; secondly, each pipeline communication link in the system needs to detect its own communication data rate; finally, when the communication data rate of the pipeline communication link reaches a threshold or it is detected that it is an important process of the system, the process using pipeline communication is migrated to the same node as the pipeline communication memory to avoid cross-node memory access and improve the efficiency of its pipeline communication. Compared with the prior art, the advantages of the present invention are: (1) Autonomous controllability: Since the software optimization and improvement are independently designed and developed, they have complete intellectual property rights.

[0038] (2) The implementation effect is obvious. Aiming at the characteristics of NUMA architecture hardware, the present invention proposes to detect the communication bandwidth of each pipeline in the system and distinguish the importance of system processes. Non-important processes with small pipeline communication bandwidth are allowed to communicate across node memory. Processes with large pipeline communication bandwidth or important processes are scheduled to communicate on the same node to increase their pipeline communication bandwidth and reduce pipeline communication delay. This method gives full play to the hardware characteristics of NUMA architecture and improves system operation efficiency.

[0039] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A pipeline communication optimization method under NUMA architecture, characterized in that: The following steps are involved: When creating a pipeline, allocate pipeline communication memory on the specified node; Detect the communication data rate of each pipeline; The threshold of each pipeline is calculated according to the type and priority of the process, and the communication data rate of each pipeline is compared with the corresponding threshold. Important processes or processes with a communication data rate greater than the threshold are migrated to the specified node for execution, while non-important processes or processes with a communication data rate less than the threshold are retained on the corresponding original node for execution.

2. The pipeline communication optimization method under the NUMA architecture according to claim 1, characterized in that: The specified node is specifically a node specified by a user, or a node running by a process for creating a pipeline.

3. The pipeline communication optimization method under the NUMA architecture according to claim 1, characterized in that: When detecting the communication data rate of each pipe, include: Get the historical communication data rate of the current pipeline, and wait for the process corresponding to the current pipeline to write data to the current pipeline or read data from the current pipeline; When a process writes data to or reads data from the current pipe, the read data or write data in the current time period in the current pipe is accumulated and recorded to obtain the communication data volume in the current time period; The communication data volume of the current time period is divided by the duration of the current time period to obtain the current communication data rate of the current pipeline; The current communication data rate is smoothed using the historical communication data rate and a preset amplification factor.

4. The pipeline communication optimization method under NUMA architecture according to claim 3, characterized in that: When the current communication data rate is smoothed using the historical communication data rate and the preset amplification factor, the expression is as follows: S = Cur_S * GF + Pre_S * (1 - GF) Where S represents the communication data rate after smoothing, Cur_S represents the current communication data rate, Pre_S represents the historical communication data rate, and GF represents the amplification factor, which ranges from (0, 1].

5. The pipeline communication optimization method under NUMA architecture according to claim 3, characterized in that: After waiting for the process corresponding to the current pipe to write data to the current pipe or read data from the current pipe, it also includes: The decay time is determined according to the priority of the process corresponding to the current pipeline. If the waiting time exceeds the decay time, the historical communication data rate is decayed using the preset decay factor to obtain the current communication data rate, and then the process corresponding to the current pipeline continues to write data to the current pipeline or read data from the current pipeline.

6. The pipeline communication optimization method under NUMA architecture according to claim 5, characterized in that: When the preset attenuation factor is used to attenuate the historical communication data rate, the expression is as follows: S = Pre_S *AF Where S represents the communication data rate after attenuation, Pre_S represents the historical communication data rate, and AF represents the attenuation factor, which ranges from (0, 1].

7. The pipeline communication optimization method under NUMA architecture according to claim 1, characterized in that: The threshold for each pipeline is calculated based on the type and priority of the process, including: Get the scheduling policy of the current process; If the scheduling policy of the current process is the real-time scheduling policy, the threshold of the pipeline corresponding to the current process is set to 0; If the scheduling policy of the current process is not a real-time scheduling policy, the priority of the current process is obtained, and the priority is substituted into the threshold calculation function to obtain the threshold of the pipeline corresponding to the current process.

8. The pipeline communication optimization method under NUMA architecture according to claim 7, characterized in that: The threshold calculation function is: pipe->threshold = PIPE_THRESHOLD * pow(a, task_nice(current)) Among them, PIPE_THRESHOLD represents the threshold when the process priority is 0, the pow(a, task_nice(current)) function implements the power operation logic of a, task_nice() is a function for obtaining the process priority, and current is the struct task_struct structure variable representing the current process.

9. The pipeline communication optimization method under NUMA architecture according to claim 7, characterized in that: Before getting the scheduling policy of the current process, it also includes: If the current process is a designated important process, the threshold of the pipeline corresponding to the current process is set to 0; If the current process is a designated unimportant process, the threshold of the pipeline corresponding to the current process is set to the maximum value of the storage type.

10. A pipeline communication optimization system under NUMA architecture, comprising a microprocessor and a computer-readable storage medium connected to each other, characterized in that: The microprocessor is programmed or configured to execute the pipeline communication optimization method under the NUMA architecture described in any one of claims 1 to 9.

Images