Stream computing engine acceleration system for single-machine multi-core architecture
By monitoring the dynamic task allocation of thread units and worker thread units under a single-machine multi-core architecture, the problem of insufficient resource utilization in the stream computing framework on the NUMA architecture is solved, achieving efficient performance improvement and cost reduction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI JIAOTONG UNIV
- Filing Date
- 2024-12-26
- Publication Date
- 2026-06-26
AI Technical Summary
Existing stream computing frameworks have not fully utilized resources and optimized performance in single-machine multi-core environments, especially in NUMA architectures where message passing and thread switching costs are high.
The system employs a monitoring thread unit and a worker thread unit. The monitoring thread unit assigns tasks based on Socket load information and combines operator properties and Socket state to achieve dynamic task allocation, thereby reducing unnecessary thread switching and message passing overhead.
It significantly improved system throughput, reduced latency, and achieved cost reduction and efficiency improvement in high-performance computing scenarios, with performance improvement ranging from 44.11% to 260.86%.
Smart Images
Figure CN122285188A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a technology in the field of high-performance computing, specifically a stream computing engine acceleration system for single-machine multi-core architectures. Background Technology
[0002] Most mainstream stream computing frameworks are designed for distributed scale-out to meet the scalability requirements of large-scale data processing. However, these systems still have significant room for improvement in resource utilization and performance optimization under single-machine multi-core environments. With the increasing demand for cost reduction and efficiency improvement, how to fully utilize local resources and reduce message passing overhead and thread switching costs on multi-core and non-uniform memory access (NUMA) architectures has become an important research direction for improving the performance and efficiency of stream computing. Summary of the Invention
[0003] To address the aforementioned shortcomings of existing technologies, this invention proposes a stream computing engine acceleration system for single-machine multi-core architectures. While retaining the scalability of general scheduling strategies, it deeply considers NUMA architecture, data dependencies between operators, and the characteristics of multi-core processors. By utilizing operator execution paths and data distribution information to dynamically adjust tasks, it reduces unnecessary thread switching, message passing, and state transition overhead. This effectively improves system throughput and reduces latency when facing real-time data streams with dynamically changing loads, ultimately achieving the goal of cost reduction and efficiency improvement in high-performance computing scenarios.
[0004] This invention is achieved through the following technical solution:
[0005] This invention relates to a stream computing engine acceleration system for single-machine multi-core architecture, comprising: a monitoring thread unit and a worker thread unit, wherein: the monitoring thread unit performs intra-Socket dispatch processing based on cross-Socket dispatch task information; the worker thread unit performs task execution processing based on specified dispatch task and intra-Socket dispatch task information respectively to obtain worker thread load results, and performs specified dispatch and cross-Socket dispatch processing based on Socket load information to obtain task dispatch results.
[0006] The monitoring thread unit includes a load monitoring module and a task dispatching module. The load monitoring module performs aggregation processing based on the worker thread load information to obtain the Socket status result. The task dispatching module performs intra-Socket dispatching processing based on cross-Socket dispatching information to obtain intra-Socket task dispatching results.
[0007] The worker thread unit includes a task execution module and a task wake-up module. The task execution module performs task execution processing based on the specified dispatched task and the task dispatched within the Socket, and obtains the worker thread load result. The task wake-up module performs specified dispatch and cross-Socket dispatch processing based on the Socket load information and the worker thread load information, and obtains the task dispatch result.
[0008] The term "Socket" refers to a memory partitioning unit in a NUMA architecture. In NUMA, memory is divided into Sockets. Sockets access local memory directly via the CPU's memory bus, while accessing remote memory requires communication via a point-to-point interconnect bus between Sockets. Remote memory access is inferior to local memory access in terms of both latency and throughput.
[0009] The aforementioned socket dispatch refers to the monitoring thread of each socket continuously checking and retrieving operators from its dispatch queue and assigning them to the worker thread with the lowest current load for execution.
[0010] The specified assignment refers to the following: when a stateless operator is woken up, the worker thread assigns the operator to an idle worker thread belonging to the same physical core to make full use of the caching acceleration effect brought by adjacent memory levels; otherwise, the operator is pushed to the monitoring thread to which the current Socket belongs.
[0011] The cross-Socket dispatch mentioned above refers to the following: when a worker thread is woken up by a stateless operator, if the worker thread under the same physical core and the current Socket have a high load, the operator will be pushed to the monitoring thread of another Socket with a lower load; when a stateful operator is woken up, if the original Socket has a high load, the operator will be pushed to the monitoring thread of another Socket with a lower load.
[0012] The stateful operator refers to a user-defined operator in stream computing that uses heap memory across a single call process and whose state memory usage is large or may increase significantly with the call process.
[0013] The stateless operator mentioned above refers to a user-defined operator in stream computing that uses only stack memory, heap memory with a lifespan of one call, or a small amount of fixed heap memory.
[0014] The load monitoring mentioned above refers to: the monitoring thread periodically monitoring the load of the worker thread queue inside the Socket, and asynchronously updating the load status of the Socket based on the number of idle threads; when dispatching across Sockets, the load at the thread level is not considered, only the load at the Socket level is considered.
[0015] The aforementioned operator execution and operator wake-up refer to the following: User-defined stream computing operators are encapsulated as asynchronous tasks in the form of coroutines, which can be passed and executed as memory objects between different threads. Operators exist in a ready-to-execute (R) state and a paused-to-execute (S) state. When new data is generated upstream or old data is consumed downstream, the operator state changes from S to R, at which point it can be assigned to a specific location, dispatched within a socket, or dispatched across sockets. When, during operator execution, upstream data consumption is complete or data accumulates downstream to a certain amount, the operator state changes from R to S and execution is paused. When the execution conditions of upstream and downstream operators are met, execution is woken up.
[0016] When the state collection involved in load monitoring is limited to within the Socket and the state information that needs to be accessed across Sockets is updated periodically rather than being updated in real time by worker threads, it can significantly reduce the problem of cache contention across Sockets and reduce scheduling overhead. Technical effect
[0017] This invention, through state-aware and NUMA-aware operator scheduling logic and socket-based delayed state updates, enables dynamic and efficient stream operator allocation based on NUMA architecture and hardware load. Compared to existing technologies, this invention significantly reduces the decision-making overhead of the scheduling algorithm itself under NUMA architecture. Attached Figure Description
[0018] Figure 1 This is a schematic diagram of the NUMA architecture;
[0019] Figure 2 This is a schematic diagram of the system of the present invention;
[0020] Figure 3 The execution flowchart for the monitoring thread;
[0021] Figure 4 Flowchart of the operation wake-up callback;
[0022] Figure 5 This is the execution flowchart for the worker thread. Detailed Implementation
[0023] like Figure 2As shown in the figure, this embodiment relates to a stream computing engine acceleration system for a single-machine multi-core architecture, including: a monitoring thread unit and a worker thread unit, wherein: the monitoring thread unit reads operators dispatched across sockets in a loop, performs internal dispatch within the socket according to the load of the worker threads within the socket, and periodically monitors the load of the worker threads within the socket and updates the overall status information of the socket; the worker thread unit continuously obtains and executes tasks from the LIFO slot and the work queue. During the execution process, other operators may be triggered to wake up, at which time the operator wake-up unit is called to dispatch the woken-up tasks.
[0024] like Figure 3 As shown, the monitoring thread unit receives operators dispatched across sockets and performs internal socket dispatching, as well as periodically updates the socket state, specifically including:
[0025] Step 1: Create a monitoring thread based on the Socket node ID and set its affinity to all logical cores on the current Socket.
[0026] Step 2: Create a Multiple Producer Single Consumer (MPSC) queue for the monitoring thread to receive operators dispatched across sockets.
[0027] Step ③: The monitoring thread executes a loop, attempting to retrieve an operator from the MPSC queue and perform internal socket dispatch execution. If no operator is retrieved within 50 milliseconds, proceed to step ④; otherwise, proceed to step ⑤.
[0028] Step 4: Iterate through the queues of all worker threads within the Socket, updating the overall Socket state based on queue load. If more than 1 / 3 of the threads are idle, update the Socket state to underloaded; otherwise, update it to loaded. Reset the task dispatch counter to 0 and return to step 3 to continue the loop.
[0029] Step 5: Traverse all work queues within the Socket and assign tasks retrieved from the MPSC queue to the first idle thread or the thread with the least load. If 50 tasks have been read and executed consecutively, proceed to step 4.
[0030] like Figure 4 As shown, the operator wake-up callback refers to the process of allocating operators based on a comprehensive consideration of the operator's own properties, the node where the last execution occurred, and the Socket load, in order to fully utilize idle computing power and reduce scheduling and execution overhead. Specifically, this includes:
[0031] Step a: Determine if the operator is a stateless operator. If yes, proceed to step b; otherwise, proceed to step e.
[0032] Step b: Determine if the current thread's socket is underloaded. If yes, proceed to step c; otherwise, proceed to step d.
[0033] Step c: Push the operator to the LIFO slot; if the slot is empty, end the operation; otherwise, retrieve the old operator from the LIFO slot. If the queue of the worker thread in which the operator is located is empty, assign the operator to the worker thread; otherwise, push the operator to the monitoring thread of the current Socket for dispatch and end this operation.
[0034] Step d: Push the operator to the least loaded socket and end this operation.
[0035] Step e: Determine if the load of the Socket where the operator last ran is higher than that of the current Socket. If yes, proceed to step d; otherwise, proceed to step f.
[0036] Step f: Push the operator back to its original socket and end this operation.
[0037] like Figure 5 As shown, the worker threads prioritize obtaining tasks from the LIFO slots, and secondly from the work queue, specifically including:
[0038] Step i: Create a corresponding number of worker threads based on the number of available logical cores in the system. Each thread is bound to an independent core.
[0039] Step ii: The thread enters the loop and checks if the termination variable is set to true. If yes, the thread exits; otherwise, it jumps to step iii.
[0040] Step iii: Attempt to retrieve a task from a LIFO slot that can only hold one operator. If the slot is empty, proceed to step iv; otherwise, proceed to step v.
[0041] Step iv: Obtain an operator from the work queue and execute it. If no task is obtained within 50 milliseconds, proceed to step ii; otherwise, proceed to step v.
[0042] Step v: Perform the task, then proceed to step ii.
[0043] Through specific practical experiments, in an environment configured with Ubuntu 18.04LTS, 2*Intel(R)Xeon(R)Gold 6230R CPUs@2.10GHz, and 2*128GB DDR4 RAM, tests were conducted using the Rust toolchain version nightly-2024-11-01. A stateful word count streaming job was selected. The dataset was excerpted from the open-source Tensorflow c4 realnewslike dataset, containing 1.04 million crawled news articles (text, links, and titles), totaling 2.8GB in size. The test results are shown in Table 1.
[0044] Table 1 This invention RisingWave model Flink model Job theft model Swallowing 17.61k / s 12.22k / s 4.88k / s 7.57k / s Year-on-year increase ———— 44.11％ 260.86％ 132.63％
[0045] Compared to existing technologies, this method accelerates message passing for stateless operators by utilizing cache resources at adjacent memory levels, reduces the overhead of state transitions for stateful operators by leveraging scheduling affinity, and mitigates cross-socket cache contention overhead for scheduling decisions in NUMA architectures by employing delayed update techniques. This effectively reduces cross-socket message passing and state transition overhead during scheduling, as well as the scheduler's cross-socket cache contention overhead. Compared to Flink's thread-based operator scheduling model, RisingWave's asynchronous operator scheduling model, and the work-stealing baseline model, the performance improvement ranges from 44.11% to 260.86%.
[0046] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.
Claims
1. A stream computing engine acceleration system for single machine multi-core architecture, characterized in that, The application relates to a task scheduling method and device. The monitoring thread unit and the working thread unit are used for: the monitoring thread unit performs Socket internal dispatching processing according to cross-Socket dispatching task information; The working thread unit respectively performs task execution processing according to specified dispatching tasks and Socket internal dispatching task information, obtains working thread load results, performs specified dispatching and cross-Socket dispatching processing according to Socket load information, and obtains task dispatching results; The Socket internal dispatching refers to the fact that the monitoring thread of each Socket continuously checks and takes out an operator from a dispatching queue of the monitoring thread and allocates the operator to a working thread with the lowest current load for execution; The specified dispatching refers to the fact that when a stateless operator is awakened, the working thread allocates the operator to an idle working thread of the same physical core to fully utilize the cache acceleration effect brought by adjacent memory levels; otherwise, the operator is pushed to the monitoring thread of the current Socket; The cross-Socket dispatching refers to the fact that when a stateless operator is awakened, if the working thread under the same physical core and the current Socket load are high, the operator is pushed to the monitoring thread of another Socket with lower load; when a stateful operator is awakened, if the load of the original Socket is too high, the operator is pushed to the monitoring thread of another Socket with lower load.
2. The stream computing engine acceleration system for single-computer multi-core architecture according to claim 1, characterized in that, The monitoring thread unit comprises a load monitoring module and a task dispatching module, wherein the load monitoring module performs aggregation processing according to working thread load information to obtain Socket state results; and the task dispatching module performs Socket internal dispatching processing according to cross-Socket dispatching information to obtain Socket internal task dispatching results.
3. The stream computing engine acceleration system for single-computer multi-core architecture according to claim 1, characterized in that, The working thread unit comprises a task execution module and a task awakening module, wherein the task execution module performs task execution processing according to specified dispatching tasks and Socket internal dispatching task information to obtain working thread load results; and the task awakening module performs specified dispatching and cross-Socket dispatching processing according to Socket load information and working thread load information to obtain task dispatching results.
4. The stream computing engine acceleration system for single-computer multi-core architecture according to claim 1, characterized in that, The stateful operator refers to a user-defined operator in stream computing, which uses heap memory across a call process in a declaration cycle and whose state memory occupation is likely to be significantly increased along with the call process; The stateless operator refers to a user-defined operator in stream computing, which only uses stack memory, heap memory with a call process in a declaration cycle or a small amount of quantitative heap memory. In the aforementioned stream computing, user-defined operators are encapsulated as asynchronous tasks in the form of coroutines, which are passed and executed between different threads as memory objects. Operators exist in a ready-to-execute (R) state and a paused-to-execute (S) state. When new data is generated upstream or old data is consumed downstream, the operator state changes from S to R, at which point it is assigned to a specific location, dispatched within a socket, or dispatched across sockets. When an operator is in execution and upstream data is consumed or data accumulates downstream to a certain amount, the operator state changes from R to S and execution is paused. When the execution conditions of upstream and downstream operators are met, execution is awakened.
5. The stream computing engine acceleration system for single-computer multi-core architecture according to claim 2, characterized in that, The load monitoring mentioned above refers to: the monitoring thread periodically monitoring the load of the worker thread queue inside the Socket, and asynchronously updating the load status of the Socket based on the number of idle threads; when dispatching across Sockets, the load at the thread level is not considered, only the load at the Socket level is considered.
6. The stream computing engine acceleration system for single-computer multi-core architecture according to claim 1 or 2 or 5, characterized in that, The monitoring thread unit receives operators dispatched across sockets and performs internal socket dispatching, as well as periodically updates the socket state, specifically including: Step 1: Create a monitoring thread based on the Socket node ID and set its affinity to all logical cores on the current Socket; Step 2: Create a multi-producer, single-consumer queue for this monitoring thread to receive operators dispatched across sockets; Step ③: Monitor the execution loop, attempt to retrieve an operator from the MPSC queue and perform internal Socket dispatch execution. If no operator is retrieved within 50 milliseconds, jump to step ④; otherwise, jump to step ⑤. Step 4: Traverse the queues of all worker threads within the Socket, and update the state of the entire Socket according to the queue load. If more than 1 / 3 of the threads are in an idle state, update the Socket state to underloaded; otherwise, update it to loaded, reset the task dispatch counter to 0, and return to step 3 to continue the loop. Step 5: Traverse all work queues in the Socket and assign the tasks taken from the MPSC queue to the first idle thread or the thread with the least load. If 50 tasks have been read and executed consecutively, jump to step 4.
7. The stream computing engine acceleration system for single-computer multi-core architecture according to claim 1 or 3 or 4, characterized in that, The aforementioned wake-up refers to: assigning operators based on a comprehensive consideration of the operator's own properties, the node where the last execution occurred, and the socket load, in order to fully utilize idle computing power and reduce scheduling and execution overhead. Specifically, this includes: Step a: Determine if the operator is a stateless operator. If yes, proceed to step b; otherwise, proceed to step e. Step b: Determine if the current thread's socket is underloaded. If so, proceed to step c; otherwise, proceed to step d. Step c: Push the operator to the LIFO slot; if the slot is empty, end; otherwise, retrieve the old operator from the LIFO slot. If the queue of the worker thread to which the operator belongs is empty, assign the operator to that worker thread; otherwise, push the operator to the monitoring thread of the current Socket for dispatch and end this operation. Step d: Push the operator to the least loaded socket and end this operation; Step e: Determine if the load of the Socket where the operator last ran is higher than that of the current Socket. If yes, proceed to step d; otherwise, proceed to step f. Step f: Push the operator back to its original socket and end this operation.
8. The stream computing engine acceleration system for single-computer multi-core architecture according to claim 7, characterized in that, The worker threads prioritize acquiring tasks from the LIFO slots, and secondarily from the work queue, specifically including: Step i: Based on the number of available logical cores in the system, create a corresponding number of worker threads, with each thread bound to an independent core; Step ii: The thread enters the loop and checks whether the termination variable is set to true. If so, the thread exits; otherwise, it jumps to step iii. Step iii: Attempt to obtain a task for execution from a LIFO slot that can only hold one operator. If the slot is empty, proceed to step iv; otherwise, proceed to step v. Step iv: Obtain an operator from the work queue and execute it. If no task is obtained within 50 milliseconds, proceed to step ii; otherwise, proceed to step v. Step v: Perform the task, then proceed to step ii.