Memory pool-based high-concurrency data processing method, device, equipment and medium

Abstract

The present application relates to the technical field of data processing, and particularly relates to a high-concurrency data processing method and device based on a memory pool, equipment and a medium, the method first acquires a predicted future hot memory specification, migrates a memory block corresponding to the predicted future hot memory specification from a global shared memory pool to a thread private memory pool, sets a resource allocation rule of the thread private memory pool, receives a data processing task, performs data block size identification processing on the data processing task to obtain a task data block size, allocates a memory block for the data processing task from the thread private memory pool based on the resource allocation rule and the task data block size, and monitors an idle memory block state of the thread private memory pool in real time, dynamically manages the thread private memory pool based on the idle memory block state and the global shared memory pool, and completes the data processing task, aiming to realize efficient memory allocation and dynamic management and control, and improve concurrency processing performance and memory utilization.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a high-concurrency data processing method, apparatus, device, and medium based on a memory pool. Background Technology

[0002] In high-concurrency data processing scenarios, memory allocation and deallocation efficiency is a core factor determining overall system performance. Traditional C++ memory allocation relies on the standard library's malloc, free, new, and delete methods, which have many unavoidable drawbacks in practical applications. In high-concurrency environments, global locks can trigger intense lock contention, directly causing thread blocking. Memory allocation can take over 50 microseconds at 100,000 QPS (Queries Per Second) concurrency, becoming a critical performance bottleneck. Frequent memory allocation and deallocation operations generate a large amount of memory fragmentation, significantly reducing memory utilization and, in severe cases, triggering memory overflow errors. This allocation method is not optimized for CPU cache lines, NUMA (Non-Uniform Memory Access) architecture, and other hardware characteristics, leading to frequent cache invalidation during memory access and further increasing data processing latency.

[0003] Meanwhile, traditional solutions lack robust exception handling mechanisms, making them highly susceptible to memory leaks due to task anomalies. Exhaustion of the memory pool under high concurrency peaks can also lead to system crashes. Furthermore, they exhibit poor adaptability to heterogeneous hardware such as FPGAs (Field Programmable Gate Arrays) and GPUs (Graphics Processing Units), failing to leverage vector instruction sets and cache allocation techniques to unleash hardware performance. C++20 coroutines lack native memory pool support, resulting in poor memory affinity and high switching overhead in concurrent scenarios. Memory pool parameters rely on manual adjustment based on experience, making it difficult to dynamically adapt to fluctuations in business traffic. Additionally, computationally intensive business rule validation consumes significant CPU resources, impacting the processing efficiency of core business logic.

[0004] Existing memory pool technologies such as tcmalloc and jemalloc only optimize lock contention issues, but do not solve core problems such as predictive optimization, hardware-coordinated offloading, deep coroutine adaptation, AI self-tuning, end-to-end observability, and business verification offloading. There is still considerable room for improvement in performance stability and scalability under extremely high concurrency scenarios. Summary of the Invention

[0005] In order to overcome the shortcomings of the prior art, the present invention aims to provide a high-concurrency data processing method, apparatus, device and medium based on memory pool, which aims to solve the problems of memory allocation lock contention, low efficiency and rough management in high-concurrency scenarios, realize efficient memory allocation and dynamic management, and improve concurrent processing performance and memory utilization.

[0006] The first aspect of this invention provides a high-concurrency data processing method based on a memory pool, wherein the memory pool includes a thread-private memory pool and a globally shared memory pool. The method includes: obtaining a predicted future hot memory specification; migrating memory blocks corresponding to the predicted future hot memory specification from the globally shared memory pool to the thread-private memory pool; and setting resource allocation rules for the thread-private memory pool; receiving a data processing task; performing data block size identification processing on the data processing task to obtain the task data block size; allocating memory blocks for the data processing task in the thread-private memory pool based on the resource allocation rules and the task data block size; and monitoring the status of idle memory blocks in the thread-private memory pool in real time; and performing dynamic memory management of the thread-private memory pool based on the status of idle memory blocks and the globally shared memory pool to complete the data processing task.

[0007] Optionally, in a first implementation of the first aspect of the present invention, obtaining the predicted future hot memory specifications includes: obtaining historical memory allocation data; and using the FTRL online learning algorithm to perform trend prediction processing on the historical memory allocation data to obtain the predicted future hot memory specifications.

[0008] Optionally, in a second implementation of the first aspect of the present invention, the step of migrating the memory block corresponding to the predicted future hot memory specification from the global shared memory pool to the thread-private memory pool and setting the resource allocation rules of the thread-private memory pool includes: extracting a free memory block of the corresponding specification from the global shared memory pool without locking according to the predicted future hot memory specification using an atomic CAS (Compare And Swap) algorithm; migrating the free memory block to the thread-private memory pool using a double-buffered lock-free atomic switching mechanism; obtaining a preset predictive latency compensation logic, adjusting the resource allocation priority of the thread-private memory pool based on the predictive latency compensation logic, and completing the setting of the resource allocation rules.

[0009] Optionally, in a third implementation of the first aspect of the present invention, the step of allocating memory blocks for the data processing task in the thread-private memory pool includes: when allocating memory blocks for the data processing task in the thread-private memory pool, using a memory allocation monitoring mechanism to monitor whether there is an abnormal state in the allocation process; when an abnormal state is detected in the allocation process, identifying abnormal memory resources; and reclaiming the abnormal memory resources to the thread-private memory pool.

[0010] Optionally, in a fourth implementation of the first aspect of the present invention, the step of dynamically managing the thread-private memory pool based on the free memory block status and the global shared memory pool to complete the data processing task includes: when the free memory block status is that the memory blocks are exhausted, using an atomic CAS algorithm to obtain supplementary memory blocks in batches from the global shared memory pool; and adding the supplementary memory blocks to the thread-private memory pool so that the thread-private memory pool has the ability to continuously allocate memory for the task, thereby completing the data processing task.

[0011] Optionally, in a fifth implementation of the first aspect of the present invention, the memory pool further includes a fragmentation reclamation pool, and the dynamic memory management of the thread-private memory pool based on the free memory block status and the global shared memory pool further includes: when the free memory block status is sufficient, performing fragmentation scanning processing on the thread-private memory pool using an FPGA parallel scanning strategy to obtain mergeable interval fragments and small fragment blocks; performing atomic merging processing on the mergeable interval fragments; performing asynchronous merging processing on the small fragment blocks through the fragmentation reclamation pool to obtain merged memory blocks; and reclaiming the merged memory blocks into the global shared memory pool.

[0012] Optionally, in a sixth implementation of the first aspect of the present invention, after completing the data processing task, the method further includes: reclaiming the memory blocks occupied by the data processing task to the corresponding memory pool, and simultaneously updating the free memory block data of the thread-private memory pool and the global shared memory pool; collecting a system operation indicator dataset, and generating real-time memory monitoring indicators based on the free memory block data and the system operation indicator dataset; inputting the real-time memory monitoring indicators into a pre-trained self-tuning model for memory pool parameter optimization processing to obtain optimized memory pool configuration parameters; and completing the self-optimization adjustment of the corresponding memory pool based on the optimized memory pool configuration parameters.

[0013] A second aspect of the present invention provides a high-concurrency data processing device based on a memory pool, wherein the memory pool includes a thread-private memory pool and a globally shared memory pool. The device comprises: a memory migration module for acquiring predicted future hot memory specifications, migrating memory blocks corresponding to the predicted future hot memory specifications from the globally shared memory pool to the thread-private memory pool, and setting resource allocation rules for the thread-private memory pool; a type extraction module for receiving data processing tasks, performing data block size identification processing on the data processing tasks to obtain the task data block size; a memory block allocation module for allocating memory blocks for the data processing tasks in the thread-private memory pool based on the resource allocation rules and the task data block size, and monitoring the status of idle memory blocks in the thread-private memory pool in real time; and a memory management module for dynamically managing the memory of the thread-private memory pool based on the status of idle memory blocks and the globally shared memory pool to complete the data processing tasks.

[0014] A third aspect of the present invention provides a memory pool-based high-concurrency data processing device, the memory pool-based high-concurrency data processing device comprising: a memory and at least one processor, the memory storing instructions; the at least one processor calling the instructions in the memory to cause the memory pool-based high-concurrency data processing device to execute the various steps of the memory pool-based high-concurrency data processing method described in any of the preceding claims.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of the memory pool-based high-concurrency data processing method described in any of the preceding claims.

[0016] In the technical solution of this invention, the predicted future hot memory specifications are first obtained, and the memory blocks corresponding to the predicted future hot memory specifications are migrated from the global shared memory pool to the thread-private memory pool. Resource allocation rules for the thread-private memory pool are then set. Next, a data processing task is received, and the data processing task undergoes data block size identification processing to obtain the task data block size. Based on the resource allocation rules and the task data block size, memory blocks are allocated from the thread-private memory pool for the data processing task. The status of idle memory blocks in the thread-private memory pool is monitored in real time. Based on the idle memory block status and the global shared memory pool, dynamic memory management is performed on the thread-private memory pool to complete the execution of the data processing task. This aims to solve the problems of memory allocation lock contention, low efficiency, and coarse management in high-concurrency scenarios, achieving efficient memory allocation and dynamic control, and improving concurrent processing performance and memory utilization. Attached Figure Description

[0017] Figure 1A flowchart illustrating the high-concurrency data processing method based on a memory pool provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a high-concurrency data processing device based on a memory pool provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a high-concurrency data processing device based on a memory pool, provided in an embodiment of the present invention. Detailed Implementation

[0018] This invention provides a high-concurrency data processing method, apparatus, device, and medium based on a memory pool. In this invention, the terms "first," "second," "third," "fourth," etc. (if present)," in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0019] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the high-concurrency data processing method based on memory pools in this invention includes: 101. Obtain the predicted future hot memory specifications, migrate the memory blocks corresponding to the predicted future hot memory specifications from the global shared memory pool to the thread-private memory pool, and set the resource allocation rules for the thread-private memory pool; In this embodiment, the global shared memory pool is a system-level fallback memory pool that holds pre-allocated large-capacity memory resources. It implements partitioned management according to memory specifications and has the ability to request memory in batches without locks and adaptively expand. The thread-private memory pool is a dedicated memory pool bound to an independent worker thread. It is built based on thread-local storage and only serves the bound thread, completely avoiding cross-thread lock contention. The system is also configured with a hierarchical sharing and isolation mechanism. At the hierarchical sharing level, read-only static data, such as basic market data configuration, is stored in the global shared pool. Process-specific data is stored in the process-private pool, and high-frequency temporary data is stored in the thread-local cache, realizing the classified storage and management of memory data at different levels. At the access control level, a read-write separation mechanism is adopted. Read operations on shared blocks are executed without locks, write operations are performed with lightweight locks, and private blocks remain completely lock-free, balancing concurrency safety while ensuring resource sharing efficiency. Predicting future hot memory specifications involves analyzing historical memory allocation data to identify memory block specifications that will be frequently used in the near future. The memory blocks corresponding to these specifications are free memory blocks in the global shared memory pool whose sizes match the predicted future hot memory specifications. The resource allocation rules are based on the thread-private memory pool resource allocation priority and scheduling strategy determined by predictive latency compensation logic.

[0020] In this embodiment, after obtaining the predicted future hot memory specifications, an atomic CAS algorithm is used to extract free memory blocks of the corresponding specifications from the global shared memory pool without locking. A double-buffered, lock-free atomic switching mechanism is then used to migrate the memory blocks to the thread-private memory pool. Finally, the resource allocation priority of the thread-private memory pool is adjusted according to a preset predictive latency compensation logic, thus completing the resource allocation rule settings. By leveraging the thread-specific characteristics of the thread-private memory pool to avoid cross-thread lock contention, and combining the partition management and lock-free batch allocation capabilities of the global shared memory pool, a predictive NUMA hotspot pre-migration mechanism is used to achieve advance scheduling of memory blocks. Simultaneously, relying on lock-free operations and priority adjustment strategies, memory allocation time is significantly reduced, memory allocation efficiency and latency stability are improved, memory resource scheduling accuracy is optimized, and traffic fluctuations in high-concurrency scenarios are adapted to ensure the atomicity of memory allocation and the stability of system operation.

[0021] 102. Receive a data processing task, perform data block size identification processing on the data processing task, and obtain the task data block size; In this embodiment, after receiving the data processing task, the data block size identification is completed through the C++ type extraction mechanism. The C++ type extraction mechanism is a compile-time template metaprogramming technique that can directly extract the core type attributes of data structures and variables during the compilation stage without runtime calculation. It can accurately parse the data structure information bound to the task and automatically calculate the number of memory bytes, thereby determining the data block size of the memory required during task execution. The obtained task data block size is a specific size value that represents the memory requirement of the data processing task. This value is used to accurately schedule the task to the partition of the thread-private memory pool of the corresponding size, so as to achieve accurate matching between the task and memory resources.

[0022] In this embodiment, the thread-private memory pool is pre-divided into fixed-size partitions of 4KB, 8KB, 16KB, 32KB, 64KB, and 128KB. The partitions are automatically matched based on the task data block size. Utilizing move semantics, memory block ownership is transferred through move constructors and assignment operators, achieving zero-copy memory block transfer. Simultaneously, the memory proportion of each sub-pool is dynamically adjusted based on real-time data traffic, allocating related data blocks of the same task to the same cache page. C++ compiler constants are used to adapt to various hardware architectures such as x86, ARM, and RISC-V, while CPU affinity binding and core isolation mechanisms reduce scheduling interference. This processing flow enables fine-grained allocation of memory resources, reduces redundant overhead in memory allocation and release, improves memory hit rate, optimizes system processing latency, enhances multi-hardware architecture adaptability, and ensures the efficiency and stability of memory allocation in high-concurrency data processing scenarios.

[0023] 103. Based on the resource allocation rules and the task data block size, allocate memory blocks for the data processing task in the thread-private memory pool, and monitor the status of free memory blocks in the thread-private memory pool in real time. In this embodiment, the thread-private memory pool is pre-divided into fixed-size partitions of 4KB, 8KB, 16KB, 32KB, 64KB, and 128KB. The memory block allocation is automatically matched to the corresponding partition based on the task data block size. The allocation process relies on move semantics to achieve zero-copy transfer of memory block ownership, allocating related data blocks of the same task to the same cache page. C++ compiler constants are used to adapt to various hardware architectures such as x86, ARM, and RISC-V. CPU affinity binding and core isolation mechanisms reduce scheduling interference. The heterogeneous concurrency adaptation layer is built based on the C++20 coroutine mechanism and CUDA memory allocation encapsulation, adapting to various concurrency modes such as coroutines and GPUs. When a coroutine is suspended, temporary memory blocks are automatically released; when it resumes, memory resources are re-allocated. Simultaneously, synchronous reclamation of the host-side and device-side memory pools is achieved. The transaction-level atomicity guarantee mechanism is based on C++ RAII (Resource Acquisition Issuance). Initialization and transactional memory context implementation integrate multi-memory block allocation operations into a unified transaction management system. The allocation process pre-allocates memory resources, and commits the allocation after all memory blocks are successfully allocated. If any allocation step fails, all pre-allocated resources are automatically rolled back. The allocation and rollback trajectory is recorded through a transactional memory log to avoid orphan blocks. During memory allocation, a memory allocation monitoring mechanism continuously monitors for abnormal states. If no memory allocation timeout or data processing task abnormality is detected, memory resources are continuously provided to the data processing task according to the established resource allocation rules. The status of free memory blocks in the thread-private memory pool is updated in real time to ensure the continuity and stability of memory allocation and support normal task execution. When a memory allocation timeout or data processing task abnormality is detected, the abnormal memory resources are quickly identified and reclaimed to the thread-private memory pool. Combined with an exception self-healing mechanism, memory leaks are avoided, ensuring the resource availability of the memory pool. The system is fully available, featuring an exception self-healing mechanism that binds memory blocks to the lifecycle of data processing tasks using custom C++ smart pointers, automatically triggering memory resource reclamation when a task fails. It also includes a preset memory allocation timeout threshold (e.g., 100μs), automatically switching to a backup memory pool built on mmap after timeout to ensure business continuity; real-time monitoring of the memory pool's running status through atomic states, automatically triggering memory fragment reclamation or memory pool expansion operations in abnormal scenarios; and a dual-clock timeout strategy to mitigate clock anomaly risks, using steady_clock to record the actual timeout time and system_clock to record the business timestamp, preventing NTP clock rollback from causing the backup memory pool to shut down erroneously. A backup memory pool, accounting for 5%-10% of total memory, is configured, with access only granted to core businesses, and combined with a tiered degradation strategy to ensure high availability of core tasks. This improves service availability in critical scenarios such as finance, fundamentally preventing memory leaks.

[0024] Throughout the entire memory allocation and scheduling process, cache coloring technology is integrated to further optimize cache resource allocation. This technology is based on Intel cache allocation technology to classify the service level of the last-level cache, achieving precise binding between the memory pool and hardware cache resources. At system startup, the last-level cache is divided into four service levels. The high-frequency small packet partition of the thread-private memory pool is bound to the service level of up to 30% of the last-level cache, the medium packet partition and the large packet partition are bound to their respective service levels, and background threads such as memory defragmentation are bound to independent service levels. All configurations are exposed to the outside world through the mba_alloc interface, and the business code can take effect without any modification. This technology can reduce the latency fluctuation of memory allocation and form a synergistic optimization effect with CPU affinity binding and core isolation mechanism, further enhancing the latency stability and resource scheduling efficiency of memory allocation in high-concurrency scenarios.

[0025] 104. Based on the status of the free memory blocks and the global shared memory pool, perform dynamic memory management on the thread-private memory pool to complete the data processing task.

[0026] In this embodiment, the system determines the status of free memory blocks in the thread-private memory pool in real time and completes node-based memory scheduling using the NUMA affinity management mechanism. During memory pool initialization, local memory pools are divided according to NUMA nodes, and processing threads corresponding to the CPU cores are bound to them. Memory resources are allocated from the local NUMA nodes first. When the free memory block status is that the memory blocks are exhausted, supplementary memory blocks are obtained in batches from the global shared memory pool through the atomic CAS algorithm and filled into the thread-private memory pool. If the local NUMA node memory resources are insufficient, memory from other nodes is borrowed with low priority and the borrowing cost is recorded. The task scheduling layer will prioritize scheduling data processing tasks to the NUMA node where the memory is located to reduce cross-node memory access overhead. For C++20 coroutine tasks, the lock-free return and reuse of free memory blocks can be achieved by relying on the coroutine stack-level memory pool. That is, the process of returning memory blocks to the coroutine stack-level memory pool does not require synchronization mechanisms such as mutexes and spinlocks, and can safely complete the return and reuse. This ensures that the thread-private memory pool has the ability to continuously allocate memory and supports the normal progress of data processing tasks. When the free memory block status indicates sufficient memory, an FPGA parallel scanning strategy is employed to perform fragmentation scanning on the thread-private memory pool. The CPU-intensive fragmentation scanning task is offloaded to the FPGA hardware for execution via the FPGA offload defragmentation module. During off-peak periods, the CPU transfers free block-related data via PCIe DMA (Peripheral Component Interconnect Express Direct Memory Access). The FPGA utilizes its parallel processing capabilities to quickly complete the scan and obtain a table of mergeable intervals. After writing back, the CPU performs an atomic merge operation in one go, using a fragmentation reclamation pool to asynchronously merge small fragmented blocks. The merged memory blocks are then reclaimed to the global shared memory pool. Simultaneously, the FPGA undertakes computationally intensive business rule verification tasks, allowing the CPU to focus on core business logic processing. This entire dynamic memory management solution combines NUMA node optimization, hardware-accelerated offloading, coroutine-specific memory adaptation, and lock-free operation technology. It can quickly fill memory gaps in the thread-private memory pool and efficiently defragment memory through hardware acceleration, significantly reducing CPU resource consumption and cross-node memory access latency, improving memory resource utilization and system concurrent processing efficiency, and providing solid support for the stable execution of high-concurrency data processing tasks.

[0027] In the process of merging and reclaiming memory blocks and allocating new memory blocks in thread-private memory pools, vector instruction acceleration technology is integrated to upgrade the performance of memory operations. This technology utilizes the RISC-V vector instruction set to specifically optimize the zero-initialization operation of memory blocks. On processor cores that support the RISC-V vector instruction set, the traditional memory block zeroing operation is replaced with a zero-vector write-back operation of vector loading and vector storage, eliminating the performance loss caused by the general zeroing function. After optimization, the zeroing bandwidth of the 4KB memory block is improved and the system power consumption is reduced. This technology, together with FPGA hardware acceleration and lock-free merging technology, forms a synergistic optimization to further improve the performance and power consumption control of the entire dynamic memory management process and maximize the execution efficiency of memory resource reuse and allocation.

[0028] In this embodiment of the invention, obtaining the predicted future hot memory specifications includes: obtaining historical memory allocation data; and using the FTRL online learning algorithm to perform trend prediction processing on the historical memory allocation data to obtain the predicted future hot memory specifications.

[0029] In this embodiment, historical memory allocation data includes real-time operational characteristics such as the allocation frequency, allocation location, NUMA node access status, and task request scale of memory blocks of various specifications within a preset time period (e.g., within 5 seconds). The FTRL online learning algorithm is an online learning algorithm designed for high-dimensional features and real-time data streams. It continuously iterates and optimizes the prediction model, accurately capturing the dynamic changes in memory allocation. By using this algorithm to perform trend prediction processing on historical memory allocation data, it can deeply analyze memory allocation characteristics and predict the memory specifications that will be frequently used within a preset time period (e.g., within 10ms), thus obtaining the predicted future hot memory specifications. Through real-time analysis and online learning prediction of historical memory allocation data, hot memory specifications in high-concurrency scenarios can be identified. Combined with a double-buffered lock-free switching mechanism, zero-blocking migration of memory blocks can be achieved, effectively solving the problem of high latency in cross-node access in NUMA architecture, significantly reducing memory allocation latency under high concurrency, improving the accuracy and response efficiency of system memory scheduling, and ensuring the stable and efficient operation of high-concurrency data processing tasks.

[0030] In this embodiment of the invention, the step of migrating the memory block corresponding to the predicted future hot memory specification from the global shared memory pool to the thread-private memory pool, and setting the resource allocation rules of the thread-private memory pool, includes: extracting a free memory block of the corresponding specification from the global shared memory pool without locking using an atomic CAS algorithm according to the predicted future hot memory specification; migrating the free memory block to the thread-private memory pool using a double-buffered lock-free atomic switching mechanism; obtaining a preset predictive latency compensation logic; adjusting the resource allocation priority of the thread-private memory pool based on the predictive latency compensation logic; and completing the setting of the resource allocation rules.

[0031] In this embodiment, based on the predicted future hot memory specifications, an atomic CAS algorithm is used to extract free memory blocks of the corresponding specifications from the global shared memory pool without locking. The atomic CAS algorithm is a lock-free atomic operation algorithm based on comparison and swap, which can safely extract memory resources in high-concurrency scenarios and avoid lock contention between threads. Free memory blocks are memory resources in the global shared memory pool that have been pre-allocated, are not occupied by tasks, and are in an available state. A double-buffered lock-free atomic switching mechanism is used to migrate the free memory blocks to the thread-private memory pool. The double-buffered lock-free atomic switching mechanism includes a main buffer A and a backup buffer B. When the memory blocks in A are exhausted, the pointer to B is atomically updated through the CAS algorithm, realizing zero-blocking operation of business threads during memory migration. A preset predictive latency compensation logic is obtained. This logic is a compensation control logic that dynamically optimizes memory allocation timing and scheduling strategy based on memory hotspot prediction results to reduce task execution latency. When it is predicted that a memory specification will be accessed frequently, the allocation of memory partitions of that specification is increased. Prioritize and respond to memory requests from corresponding tasks. When it is predicted that the task execution latency is about to exceed the threshold, increase the scheduling weight of the associated thread's private memory pool and prioritize the allocation of local NUMA node memory resources. When it is predicted that the probability of cross-node memory access will increase, increase the resource priority of the target NUMA node's thread's private memory pool to reduce the latency loss caused by cross-node access. When it is predicted that business traffic will surge, increase the priority of the memory partition corresponding to the core business and restrict the memory preemption priority of non-core businesses. When it is predicted that memory allocation competition will intensify, increase the priority of batch replenishment of hot memory specifications to ensure the stability of memory supply. Based on the predictive latency compensation logic, adjust the resource allocation priority of the thread's private memory pool according to the above rules. The resource allocation priority is the execution order of memory allocation set according to the memory hotspot attributes and latency compensation requirements. Finally, the resource allocation rules are set. The resource allocation rules are a standardized memory allocation and scheduling specification formed by integrating allocation priority, memory partition matching rules, and multi-architecture adaptation strategies. By leveraging lock-free operations and predictive scheduling, it achieves precise pre-allocation and migration of memory resources, dynamically optimizes resource allocation strategies, significantly reduces memory allocation latency and scheduling interference, improves memory hit rate and latency stability, provides underlying support for precise binding of tasks and memory, and adapts to memory scheduling needs in high-concurrency data processing scenarios.

[0032] In this embodiment of the invention, allocating memory blocks for the data processing task in the thread-private memory pool includes: when allocating memory blocks for the data processing task in the thread-private memory pool, using a memory allocation monitoring mechanism to monitor whether there is an abnormal state in the allocation process; when an abnormal state is detected in the allocation process, identifying abnormal memory resources; and reclaiming the abnormal memory resources to the thread-private memory pool.

[0033] In this embodiment, memory blocks are allocated to data processing tasks from a thread-private memory pool based on resource allocation rules and task data block sizes. A memory allocation monitoring mechanism is employed to continuously monitor for abnormal states during the allocation process. This mechanism integrates atomic state monitoring, lifecycle binding, and timeout determination into a comprehensive protection technology. It binds memory blocks to the task's lifecycle using custom smart pointers, monitoring memory allocation time, task execution status, and memory pool operating parameters in real time. Dual-clock timing logic is also used to mitigate monitoring biases caused by clock anomalies. Memory allocation timeout refers to the execution time of the memory allocation operation exceeding a preset timeout threshold. Data processing task anomalies refer to states where execution errors, transaction interruptions, or resource preemption failures prevent the task from progressing normally. When no anomalies are detected, the memory allocation process is completed using transaction-level atomicity guarantee technology. This technology, based on the C++ RAII mechanism, constructs a unified transaction memory context, centrally managing all memory operations within a single transaction. During the allocation phase, memory resources are pre-allocated, and the process is committed after all memory blocks are successfully allocated, ensuring the integrity of memory allocation. When a memory allocation timeout or data processing task anomaly is detected, the abnormal memory resources are quickly locked and identified. All pre-allocated memory resources are rolled back in batches using transaction rollback technology, and the abnormal memory resources are reclaimed to the thread-private memory pool. In the case of allocation timeout, the system automatically switches to the backup memory pool based on mmap (Memory Map), and triggers tiered degradation and backup memory scheduling logic. The system reserves 5% to 10% of the total memory as backup memory for core business use, and prioritizes suspending memory allocation requests for non-core business. In abnormal states, the system automatically triggers memory fragmentation reclamation or memory pool expansion operations. The dual-clock timing technology uses a steady-state clock to record the actual timeout time and a system clock to record the business timestamp, avoiding the accidental shutdown of the backup memory pool caused by NTP (Network Time Protocol) clock rollback. The entire solution provides full-process anomaly protection and automated resource reclamation for memory allocation. It eliminates partial success issues in multi-memory block allocation by relying on transaction atomicity guarantees, and ensures uninterrupted operation of core businesses through multi-level fallback and graded degradation strategies. It solves the problems of memory leaks caused by task anomalies and business interruptions caused by allocation timeouts, and significantly improves the robustness of system memory management and the operational stability in high-concurrency scenarios.

[0034] In this embodiment of the invention, the step of dynamically managing the thread-private memory pool based on the free memory block status and the global shared memory pool to complete the data processing task includes: when the free memory block status is that the memory blocks are exhausted, using an atomic CAS algorithm to obtain supplementary memory blocks in batches from the global shared memory pool; and adding the supplementary memory blocks to the thread-private memory pool so that the thread-private memory pool has the ability to continuously allocate memory for the task, thereby completing the data processing task.

[0035] In this embodiment, the system determines the memory block exhaustion state by real-time monitoring of the remaining number and available capacity of free memory blocks in the thread-private memory pool. When the remaining free memory blocks cannot meet the memory allocation requirements of the currently executing and scheduled data processing tasks, the free memory block state is determined to be memory block exhaustion. The atomic CAS algorithm, as a lock-free atomic operation technology based on comparison and swap, can avoid thread lock contention in high-concurrency environments, ensuring the safety and efficiency of memory resource extraction. The global shared memory pool serves as the core backup memory resource carrier of the system. During initialization, it divides the local memory pool according to NUMA nodes and binds it to the processing thread of the corresponding CPU core. When batch acquiring supplementary memory blocks, it prioritizes allocating memory resources from the local NUMA nodes. If the local node memory resources are insufficient, it allocates memory resources with lower priority. The system borrows memory from other nodes and records the borrowing cost. At the same time, the task scheduling layer prioritizes scheduling data processing tasks to the NUMA nodes where the memory resides, reducing the performance loss caused by cross-node memory access. If the global shared memory pool resources are insufficient during batch acquisition, the system automatically triggers an adaptive expansion strategy to improve memory supply capacity. If the allocation demand still cannot be met after expansion, a tiered degradation mechanism is activated and a backup memory pool is enabled to ensure memory allocation for core data processing tasks. The acquired supplementary memory blocks are directly added to the thread-private memory pool. The thread-private memory pool uses C++ type extraction to identify the size of task-related data blocks, matches the corresponding memory partitions, and achieves zero-copy transfer of memory blocks through move semantics, continuously providing stable memory allocation capabilities for data processing tasks and ensuring smooth task execution. This memory replenishment mechanism combines lock-free atomic operations, NUMA affinity scheduling, and multi-level fallback strategies to effectively solve the task interruption problem caused by the exhaustion of thread-private memory pools, eliminate lock contention overhead in high-concurrency scenarios, reduce cross-node memory access latency, and improve the system's resilience to traffic fluctuations through adaptive expansion and tiered degradation mechanisms, ensuring the continuity and stability of memory allocation and providing solid support for the continuous and efficient execution of high-concurrency data processing tasks.

[0036] In this embodiment of the invention, the memory pool further includes a fragmentation reclamation pool. The dynamic memory management of the thread-private memory pool based on the free memory block status and the global shared memory pool further includes: when the free memory block status is sufficient, performing fragmentation scanning processing on the thread-private memory pool using an FPGA parallel scanning strategy to obtain mergeable interval fragments and small fragment blocks; performing atomic merging processing on the mergeable interval fragments; performing asynchronous merging processing on the small fragment blocks through the fragmentation reclamation pool to obtain merged memory blocks; and reclaiming the merged memory blocks into the global shared memory pool.

[0037] In this embodiment, the system monitors the total amount of free memory blocks in the thread-private memory pool and the available capacity of each partition in real time. When the free memory resources far exceed the memory allocation requirements of the current and pending data processing tasks, and all memory partitions retain sufficient allocable memory blocks, the system determines that the free memory block status is sufficient. The fragmentation pool is a management unit dedicated to the consolidation of discrete memory fragments. It marks mergeable memory blocks through the C++ RAII mechanism, determines the business off-peak period based on the concurrency threshold, and starts a background thread to asynchronously execute fragment merging without blocking the business thread. At the same time, it records the adjacent free states of each memory block, merges consecutive free blocks into large-size memory blocks, and reuses them, effectively reducing the memory fragmentation rate. It also supports working in conjunction with the FPGA offload defragmentation module to further improve fragmentation processing efficiency. The FPGA parallel scanning strategy is an accelerated processing strategy that offloads CPU-intensive fragmentation scanning tasks to FPGA hardware execution. During off-peak hours, the CPU pushes the free block list head pointer and length array to the FPGA via PCIe DMA. The FPGA's parallel processing and pipelined execution capabilities enable high-speed scanning, significantly reducing scanning time without consuming CPU core resources. After scanning, mergeable interval fragments and small fragment blocks are obtained. Mergeable interval fragments are combinations of contiguous free memory blocks in a thread-private memory pool, possessing the physical conditions to be directly merged into larger memory blocks. Small fragment blocks are discrete, small-sized, and fragmented free memory blocks without contiguous space. The core difference between the two is their spatial distribution: mergeable interval fragments are contiguous and can be directly merged, while small fragment blocks are discrete and require regularization before merging. The FPGA uses parallel scanning to quickly output a mergeable table. After receiving the mergeable interval table from the FPGA, the CPU completes the merging operation of contiguous memory blocks in one go through lock-free atomic operations, without blocking the business thread. Asynchronous merging processing is implemented using a fragmentation reclamation pool, using C++. The RAII mechanism marks small fragmented blocks and identifies off-peak periods based on system concurrency thresholds. An independent background thread is then initiated to asynchronously merge these discrete fragments without interfering with the normal execution of foreground services. After merging, standardized merged memory blocks are formed. The system then uses a lock-free data interaction mechanism to uniformly reclaim these merged memory blocks into the global shared memory pool, incorporating them into the global memory resource partitioning management system. This provides continuous and stable memory replenishment for thread-private memory pools, ensuring that these pools continuously provide memory allocation services for data processing tasks and support their complete execution. This fragmentation processing solution combines hardware acceleration, lock-free atomic operations, and asynchronous background processing technology to significantly improve fragmentation efficiency, reduce system memory fragmentation rate, free up CPU core resources for focused business processing, and simultaneously achieve memory resource reuse, greatly improving overall memory utilization and ensuring the efficiency and stability of memory allocation in high-concurrency data processing scenarios.

[0038] In this embodiment of the invention, after completing the data processing task, the method further includes: reclaiming the memory blocks occupied by the data processing task to the corresponding memory pool, and simultaneously updating the free memory block data of the thread-private memory pool and the global shared memory pool; collecting a system operation indicator dataset, and generating real-time memory monitoring indicators based on the free memory block data and the system operation indicator dataset; inputting the real-time memory monitoring indicators into a pre-trained self-tuning model for memory pool parameter optimization processing to obtain optimized memory pool configuration parameters; and completing the self-optimization adjustment of the corresponding memory pool based on the optimized memory pool configuration parameters.

[0039] In this embodiment, after the data processing task is completed, the memory blocks occupied by the data processing task are automatically reclaimed to the corresponding memory pool for reuse by subsequent tasks. After memory reclamation is completed, the free memory block data of the thread-private memory pool and the global shared memory pool are updated synchronously. The update process uses atomic operations to correct the number of available memory blocks, partition capacity, and occupancy status of the memory pool in real time, and synchronously records the reclamation location and specification information of the memory blocks to ensure accurate synchronization of memory resource status. A system operation indicator dataset is collected, which includes full-dimensional operation data such as QPS, memory fragmentation rate, cache hit / miss ratio, CPU utilization, P99 latency, and number of concurrent conflicts. The eBPF (extended BerkeleyPacket Filter) kernel-mode non-intrusive monitoring technology is a monitoring technology that completes data collection in kernel mode by hooking the memory pool allocation and release interface. It can avoid the business intrusion and performance loss caused by user-mode monitoring. Based on the free memory block data and the system operation indicator dataset, real-time memory monitoring indicators are generated using this technology. The real-time memory monitoring indicators include key data such as memory specification hotspot distribution and fragmentation concentration areas, and data output is completed at a second-level frequency with extremely low overall performance loss. Real-time memory monitoring metrics are input into a pre-trained self-tuning model for memory pool parameter optimization. The pre-trained self-tuning model is built using a lightweight ONNX (Open Neural Network Exchange) architecture, with a model size of less than 200KB. The training process uses historical system operation metrics and memory pool status data as training samples to complete model parameter convergence and performance verification. After deployment, metric inference calculations are performed every 5 seconds to obtain optimized memory pool configuration parameters. The optimized memory pool configuration parameters include core configurations such as the maximum length of the free list, batch allocation size, defragmentation interval, memory pool capacity, and number of worker threads. It can dynamically trigger memory defragmentation, switch lock-free operation modes, and adjust the amount of pre-allocated memory based on the system's operating status. At the same time, it works with the LLVM Pass tool to automatically identify performance bottlenecks such as cache misses and branch prediction failures and correct the parameters to complete the self-optimization and adjustment of the memory pool. It achieves automated memory resource reclamation and closed-loop optimization. Relying on eBPF observability technology for non-intrusive and highly reliable monitoring, lightweight AI model for real-time inference, and LLVM (Low Level Virtual Machine) automatic performance profiling technology, it eliminates the dependence on manual parameter adjustment, reduces data processing latency, and continuously improves memory resource utilization and system stability, providing dynamically adapted and continuously optimized underlying memory support for high-concurrency data processing scenarios.

[0040] The high-concurrency data processing method based on a memory pool in the embodiments of the present invention has been described above. The high-concurrency data processing apparatus based on a memory pool in the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 2One embodiment of the high-concurrency data processing device based on a memory pool in this invention includes: Memory migration module 201: used to obtain the predicted future hot memory specifications, migrate the memory blocks corresponding to the predicted future hot memory specifications from the global shared memory pool to the thread private memory pool, and set the resource allocation rules of the thread private memory pool; Type extraction module 202: used to receive data processing tasks, perform data block size identification processing on the data processing tasks, and obtain the task data block size; Memory block allocation module 203: Based on the resource allocation rules and the task data block size, it allocates memory blocks for the data processing task in the thread-private memory pool, and monitors the status of free memory blocks in the thread-private memory pool in real time. Memory management module 204: used to dynamically manage the thread-private memory pool based on the status of the free memory blocks and the global shared memory pool, so as to complete the data processing task.

[0041] Based on the same ideas as the methods in the above embodiments, the apparatus provided in this application can implement the methods in the above embodiments.

[0042] above Figure 2 The high-concurrency data processing device based on memory pool in this embodiment of the invention is described in detail from the perspective of modular functional entities. The high-concurrency data processing device based on memory pool in this embodiment of the invention is described in detail from the perspective of hardware processing.

[0043] Figure 3 This is a schematic diagram of the structure of a memory pool-based high-concurrency data processing device 300 provided in an embodiment of the present invention. The memory pool-based high-concurrency data processing device 300 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the memory pool-based high-concurrency data processing device 300. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute a series of instruction operations on the memory pool-based high-concurrency data processing device 300 to implement the steps of the memory pool-based high-concurrency data processing method provided in the above-described method embodiments.

[0044] The memory pool-based high-concurrency data processing device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The illustrated high-concurrency data processing device architecture based on memory pools does not constitute a limitation on high-concurrency data processing devices based on memory pools. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0045] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of a memory pool-based high-concurrency data processing method.

[0046] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0047] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0048] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A high-concurrency data processing method based on a memory pool, characterized in that, The memory pool includes a thread-private memory pool and a globally shared memory pool, and the method includes: Historical memory allocation data is acquired, and the historical memory allocation data is processed by the FTRL online learning algorithm to predict future hot memory specifications. The memory blocks corresponding to the predicted future hot memory specifications are migrated from the global shared memory pool to the thread-private memory pool, and the resource allocation rules of the thread-private memory pool are set. This includes: extracting free memory blocks of the corresponding specifications from the global shared memory pool without locking using an atomic CAS algorithm based on the predicted future hot memory specifications; migrating the free memory blocks to the thread-private memory pool using a double-buffered lock-free atomic switching mechanism; acquiring a preset predictive latency compensation logic; adjusting the resource allocation priority of the thread-private memory pool based on the predictive latency compensation logic; and completing the setting of the resource allocation rules. Receive a data processing task, perform data block size identification processing on the data processing task, and obtain the task data block size; Based on the resource allocation rules and the task data block size, when allocating memory blocks for the data processing task in the thread-private memory pool, a memory allocation monitoring mechanism is used to monitor whether there is an abnormal state in the allocation process. When an abnormal state is detected in the allocation process, abnormal memory resources are identified, and the abnormal memory resources are reclaimed to the thread-private memory pool. The status of free memory blocks in the thread-private memory pool is monitored in real time. Based on the free memory block status and the global shared memory pool, the thread-private memory pool is dynamically managed to complete the data processing task. This includes: when the free memory block status is that the memory blocks are exhausted, using an atomic CAS algorithm to obtain supplementary memory blocks in batches from the global shared memory pool, and adding the supplementary memory blocks to the thread-private memory pool, so that the thread-private memory pool has the ability to continuously allocate memory for the task, thereby completing the data processing task.

2. The high-concurrency data processing method based on a memory pool according to claim 1, characterized in that, The memory pool also includes a fragmentation reclamation pool. The dynamic memory management of the thread-private memory pool based on the free memory block status and the global shared memory pool further includes: When the free memory block is in a state of sufficient memory blocks, the FPGA parallel scanning strategy is used to perform fragmentation scanning on the thread private memory pool to obtain mergeable interval fragments and small fragment blocks. The mergeable interval fragments are then subjected to atomic merging. The small fragmented blocks are asynchronously merged using the fragmentation recycling pool to obtain merged memory blocks. The merged memory blocks are recycled back to the global shared memory pool.

3. The high-concurrency data processing method based on a memory pool according to claim 1, characterized in that, After completing the data processing task, the process further includes: The memory blocks occupied by the data processing task are reclaimed to the corresponding memory pool, and the free memory block data of the thread-private memory pool and the global shared memory pool are updated. Collect system operation indicator datasets, and generate real-time memory monitoring indicators based on the free memory block data and the system operation indicator datasets; The real-time memory monitoring metrics are input into a pre-trained self-tuning model to optimize the memory pool parameters, resulting in optimized memory pool configuration parameters. Based on the optimized memory pool configuration parameters, the corresponding memory pool is self-optimized and adjusted.

4. A high-concurrency data processing device based on a memory pool, characterized in that, The memory pool includes a thread-private memory pool and a globally shared memory pool, and the device includes: Memory migration module: Used to acquire historical memory allocation data, perform trend prediction processing on the historical memory allocation data using the FTRL online learning algorithm to obtain the predicted future hot memory specifications, acquire the predicted future hot memory specifications, migrate the memory blocks corresponding to the predicted future hot memory specifications from the global shared memory pool to the thread-private memory pool, and set the resource allocation rules of the thread-private memory pool, including: according to the predicted future hot memory specifications, using an atomic CAS algorithm to extract the corresponding free memory blocks from the global shared memory pool without locks, using a double-buffered lock-free atomic switching mechanism to migrate the free memory blocks to the thread-private memory pool, acquire preset predictive latency compensation logic, adjust the resource allocation priority of the thread-private memory pool based on the predictive latency compensation logic, and complete the setting of resource allocation rules; Type extraction module: used to receive data processing tasks, perform data block size identification processing on the data processing tasks, and obtain the task data block size; Memory block allocation module: When allocating memory blocks for the data processing task in the thread-private memory pool based on the resource allocation rules and the task data block size, the module uses a memory allocation monitoring mechanism to monitor whether there is an abnormal state in the allocation process. When an abnormal state is detected in the allocation process, the module identifies the abnormal memory resources, reclaims the abnormal memory resources to the thread-private memory pool, and monitors the status of free memory blocks in the thread-private memory pool in real time. Memory management module: used to dynamically manage the memory of the thread-private memory pool based on the status of the free memory blocks and the global shared memory pool to complete the data processing task, including: when the free memory block status is that the memory blocks are exhausted, using an atomic CAS algorithm to obtain supplementary memory blocks in batches from the global shared memory pool, and adding the supplementary memory blocks to the thread-private memory pool, so that the thread-private memory pool has the ability to continuously allocate memory for the task, thereby completing the data processing task.

5. A high-concurrency data processing device based on a memory pool, characterized in that, The memory pool-based high-concurrency data processing device includes: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors invokes the instructions in the memory to cause the memory pool-based high-concurrency data processing device to perform the steps of the memory pool-based high-concurrency data processing method as claimed in any one of claims 1-3.

6. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the steps of the memory pool-based high-concurrency data processing method as described in any one of claims 1-3.

Images