A multi-level in-memory computing and storage fusion architecture and pipeline implementation method

Abstract

The application discloses a multi-level heterogeneous storage-computing fusion architecture and a pipeline implementation method, comprising the following steps: receiving a service task and analyzing and splitting through a CPU general computing layer; delivering a parallel computing task to an accelerator card special computing layer and delivering a data quality task to a storage computing layer; performing core computing of the accelerator card layer to generate standardized intermediate data and pushing the standardized intermediate data to a storage layer; cleaning and verifying the intermediate data by using a local embedded computing unit of the storage layer; converging and outputting results of the CPU layer to form a single pipeline closed loop; updating a pipeline state and scheduling a next round of tasks to realize flow series processing; meanwhile, the accelerator card layer performs parallel core computing of the next batch, and the CPU layer completes parallel task analysis and resource pre-scheduling of the next batch. The application realizes three-level storage-computing deep fusion, reduces data cross-layer transportation, reduces transmission delay, improves computing power utilization rate and system throughput performance, and balances single task processing precision and multi-batch parallel high-throughput demand.

Description

Technical Field

[0001] This application relates to the field of CPU architecture technology, and more specifically, to a multi-level heterogeneous in-memory computing architecture and pipeline implementation method. Background Technology

[0002] With the rapid development of technologies such as artificial intelligence and big data analytics, the scale of data and computational complexity that high-performance computing systems need to process have increased dramatically. To improve computing power, heterogeneous computing architectures have gradually become mainstream, accelerating computing tasks through the collaborative work of CPUs and accelerator cards. However, traditional high-performance computing systems generally adopt a storage-compute separation architecture, where the CPU, accelerator card, and independent storage are separated. Computing units and storage units are deployed independently, and data needs to be frequently migrated between system memory, accelerator card memory, and external storage media. This architecture has gradually revealed serious performance bottlenecks when facing modern computing scenarios with high throughput, low latency, and high resource utilization.

[0003] Currently, mainstream heterogeneous collaborative solutions mostly consist of a two-tier computing architecture composed of CPUs and accelerator cards, combined with independent storage devices to form a storage-compute separation system. This existing technology suffers from the following key shortcomings: First, storage-compute separation leads to frequent data interaction between different storage levels, consuming significant bus bandwidth, resulting in high transmission latency and severely restricting the overall computing power release of the system. Second, computing resource utilization is low; relying solely on two-tier collaboration between CPUs and accelerator cards fails to leverage lightweight computing resources at the storage end, leaving underlying computing power idle for extended periods and resulting in severe resource redundancy. Third, the collaborative scheduling mode is rigid, supporting only simple intra-level parallel or single-level serial jobs, unable to achieve serial connection of multi-level architecture pipelines and cross-level parallel collaboration, lacking scheduling flexibility and struggling to adapt to complex computing tasks with multiple granularities and types. Fourth, job chain breaks occur; data preprocessing, core computation, and storage deployment processes are disconnected, lacking a closed-loop pipeline of control, acceleration, and storage computing, limiting the overall system throughput efficiency. These shortcomings make the existing architecture unable to meet the current demands for high throughput, low latency, and high resource utilization in computing services. Summary of the Invention

[0004] In view of at least one defect or improvement requirement of the prior art, this application provides a multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method, which can solve at least one of the problems existing in the above background art.

[0005] To achieve the above objectives, according to the first aspect of this application, a multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method are provided, the method comprising: The CPU general computing layer receives business tasks, parses and breaks them down into hierarchical execution instructions; The CPU general computing layer distributes parallel computing tasks to the accelerator card dedicated computing layer, and the data quality processing tasks are distributed to the storage computing layer. Core computations are performed through the dedicated computing layer of the accelerator card, generating standardized intermediate data and pushing it to the storage computing layer; The storage computing layer utilizes local embedded computing units to clean, optimize the format, and verify the consistency of the standardized intermediate data, thereby completing data regularization and persistent storage. The CPU general computing layer aggregates and outputs the processing results to form a single pipeline closed loop. The pipeline state is updated through the general computing layer of the CPU, and the next round of tasks is dynamically scheduled to achieve streaming serial processing. While the storage computing layer processes the current batch of data organization and landing tasks, the accelerator card dedicated computing layer executes the next batch of core computing tasks in parallel, and the CPU general computing layer completes the parsing and resource pre-scheduling of the next batch of tasks in parallel.

[0006] Furthermore, the aforementioned multi-level heterogeneous in-memory computing fusion architecture and pipeline implementation method also includes a scenario-adaptive dynamic scheduling step: Based on the task decomposition rule base, the business tasks are divided into lightweight general-purpose tasks, computing power accelerated tasks, and high-concurrency tasks according to their functional attributes. Label all tasks with priority tags, resource level tags, and timeliness tags to generate a standardized task distribution list; Automatically switch job modes based on task type and real-time system load status: For lightweight general-purpose tasks, a two-level collaborative mode of CPU and storage computing layers is enabled; for computing power-accelerated tasks, a three-level full-link serial pipeline mode is enabled; and for massive concurrent tasks, a multi-level parallel collaborative mode is activated.

[0007] Furthermore, in the above-mentioned multi-level heterogeneous storage-computing convergence architecture and pipeline implementation method, the storage-computing layer integrates an embedded field-programmable gate array or a smart solid-state drive, and the local embedded computing unit is configured to perform data pre-processing and post-processing locally on the storage medium, so that the data is cleaned, normalized and verified in the storage layer before being transmitted outward.

[0008] Furthermore, in the above-mentioned multi-level heterogeneous in-memory computing convergence architecture and pipeline implementation method, the dedicated computing layer of the accelerator card adopts a multi-accelerator card array cluster deployment, and multiple accelerator cards work together in parallel. The CPU general computing layer is further configured to dynamically adjust the number of accelerator cards participating in the computation and the task allocation ratio according to the data scale and computing power requirements of the task.

[0009] Furthermore, in the aforementioned multi-level heterogeneous in-memory computing convergence architecture and pipeline implementation method, the unified timing synchronization mechanism includes: Assign a globally unique batch identifier to each batch of tasks; Each level in the three-tier architecture carries the batch identifier when processing tasks, and verifies the consistency of the batch identifier when the task is transmitted.

[0010] Furthermore, in the above-mentioned multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method, the lightweight general-purpose task includes at least one of process scheduling, logical judgment, permission verification, exception handling and result integration; The computing power acceleration tasks include at least one of matrix operations, model reasoning, and high-density floating-point computing. The massive concurrent tasks include at least one of the following: data cleaning, deduplication, format standardization, aggregation statistics, and pre-computation.

[0011] Furthermore, the aforementioned multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method also includes fault-tolerant processing steps: When any level detects a task execution anomaly, the anomaly information is reported to the CPU general computing layer; The CPU general computing layer performs task retry, task migration to a backup computing unit, or downgrade processing according to the exception type, and updates the priority tag and resource level tag in the standardized task distribution list.

[0012] According to a second aspect of this application, a multi-level heterogeneous in-memory computing converged architecture and pipeline implementation device are also provided, comprising: The service receiving module is used to receive service tasks through the CPU general computing layer, parse and split them into hierarchical execution instructions; The task distribution module is used to distribute parallel computing tasks to the accelerator card dedicated computing layer through the CPU general computing layer, and to distribute data quality processing tasks to the storage computing layer. The core computing module is used to perform core computing through the dedicated computing layer of the accelerator card, generate standardized intermediate data, and push it to the storage computing layer; The data processing module is used to clean, optimize the format, and verify the consistency of the standardized intermediate data through the storage computing layer using a local embedded computing unit, thereby completing data regularization and persistent storage. The result output module is used to aggregate and output the processing results through the CPU general computing layer to form a single pipeline closed loop. The state update module is used to update the pipeline state through the CPU general computing layer and dynamically schedule the next round of tasks to achieve streaming serial processing. The parallel computing module is used to simultaneously execute the next batch of core computing tasks in parallel by the accelerator card dedicated computing layer while the storage computing layer processes the current batch of data regularization and landing tasks, and the CPU general computing layer completes the parsing and resource pre-scheduling of the next batch of tasks in parallel.

[0013] According to a third aspect of this application, a multi-level heterogeneous in-memory computing architecture and pipeline implementation device is also provided, which includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit performs the steps of any of the methods described above.

[0014] According to a fourth aspect of this application, a storage medium is also provided, which stores a computer program executable by a multilevel heterogeneous in-memory computing architecture and pipeline implementation device, wherein when the computer program is run on the multilevel heterogeneous in-memory computing architecture and pipeline implementation device, the multilevel heterogeneous in-memory computing architecture and pipeline implementation device performs the steps of any of the methods described above.

[0015] In summary, compared with the prior art, the above-described technical solutions conceived in this application can achieve the following beneficial effects: The multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method provided in this application can overcome the limitations of traditional two-level heterogeneous architectures and in-memory computing separation architectures. By constructing a three-level hierarchical topology structure of a CPU general computing layer, an accelerator card dedicated computing layer, and a storage computing layer, it establishes a closed-loop in-memory computing pipeline that connects the entire "control-acceleration-storage computing" chain. This effectively reduces the frequent cross-level migration of data between memory, video memory, and storage media, and lowers bus transmission latency and bandwidth consumption. Simultaneously, through a dual-mode cooperation of serial connection of the three-level pipeline and staggered batch parallel collaboration, each level can operate independently and without blocking each other when processing different batches of tasks, achieving uninterrupted task streaming processing and significantly improving the overall system throughput performance and computing resource utilization. Furthermore, by sinking lightweight computing capabilities to the storage end, it fully utilizes the idle computing power of the storage layer, avoids resource redundancy and waste, and balances the integrity of single-task processing with the ultra-high throughput requirements of multi-batch parallel processing, significantly enhancing the system's adaptability to complex computing scenarios. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1A flowchart illustrating a multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method provided in this application embodiment; Figure 2 This is a schematic diagram of a multi-level heterogeneous in-memory computing converged architecture design provided in an embodiment of this application; Figure 3 This is a schematic diagram of the overall workflow provided for an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. Furthermore, the technical features involved in the various embodiments described below can be combined with each other as long as they do not conflict with each other.

[0019] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0020] Figure 1 This application provides a flowchart illustrating a multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method, as shown in the embodiments below. Figure 1 As shown in the figure, the multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method provided in this application embodiment includes the following steps: The CPU general computing layer receives business tasks, parses and breaks them down into hierarchical execution instructions; The CPU general computing layer distributes parallel computing tasks to the accelerator card dedicated computing layer, and the data quality processing tasks are distributed to the storage computing layer. Core computations are performed through the dedicated computing layer of the accelerator card, generating standardized intermediate data and pushing it to the storage computing layer; The storage computing layer utilizes local embedded computing units to clean, optimize the format, and verify the consistency of the standardized intermediate data, thereby completing data regularization and persistent storage. The CPU general computing layer aggregates and outputs the processing results to form a single pipeline closed loop. The pipeline state is updated through the general computing layer of the CPU, and the next round of tasks is dynamically scheduled to achieve streaming serial processing. While the storage computing layer processes the current batch of data organization and landing tasks, the accelerator card dedicated computing layer executes the next batch of core computing tasks in parallel, and the CPU general computing layer completes the parsing and resource pre-scheduling of the next batch of tasks in parallel.

[0021] Specifically, this application discloses a multi-level heterogeneous in-memory computing architecture and pipeline implementation method, applied to a three-level hierarchical heterogeneous topology system comprising a CPU general-purpose computing layer, an accelerator card dedicated computing layer, and a storage computing layer. The CPU general-purpose computing layer serves as the top-level control core, the accelerator card dedicated computing layer as the middle-level computing power core, and the storage computing layer as the bottom-level integrated in-memory computing core. The three-level architecture constructs an integrated topology link through a high-speed interconnect bus.

[0022] The CPU general computing layer receives business tasks from the upper layer, performs task parsing, granular decomposition and priority sorting, and generates hierarchical execution instructions adapted to the three-level architecture, providing a unified scheduling benchmark for subsequent cross-level collaboration.

[0023] The CPU general-purpose computing layer dynamically allocates accelerator card computing resources and storage computing layer job resources based on task computing power requirements and data scale. Specifically, parallel computing tasks are distributed to the accelerator card dedicated computing layer, data quality processing tasks are distributed to the storage computing layer, and complex control tasks are executed autonomously by the CPU general-purpose computing layer.

[0024] After receiving the structured task, the dedicated computing layer of the accelerator card initializes the computing unit, performs high-density parallel computing such as matrix operations, model inference or feature extraction, generates standardized intermediate data, and pushes the intermediate data to the storage computing layer.

[0025] The storage and computing layer integrates high-speed storage media and embedded computing units, breaking through the limitations of traditional storage which only allows reading and writing. Through local embedded computing units, the storage and computing layer performs data cleaning, format optimization, and consistency verification on the received standardized intermediate data, followed by data regularization and persistent storage. This enables local computation and local storage of data, reducing redundant data movement across layers.

[0026] The dedicated computing layer and storage computing layer of the accelerator card send the processing results back to the general computing layer of the CPU. The general computing layer of the CPU then completes the final result aggregation and business output, forming a single pipeline closed-loop operation.

[0027] The CPU general computing layer updates the pipeline state and dynamically schedules the next round of pipeline tasks. Each layer continuously iterates and executes the above process to achieve uninterrupted serial processing of tasks.

[0028] Furthermore, the method in this application also includes multi-level parallel collaborative operation steps: while the storage computing layer processes the current batch of data preparation and write-down tasks, the accelerator card dedicated computing layer executes the next batch of core computing tasks in parallel, and the CPU general-purpose computing layer completes the parsing and resource pre-scheduling of the next batch of tasks in parallel. Specifically, the system initiates a batch staggered parallel mechanism. While the storage computing layer processes the Nth batch of data preparation and write-down tasks, the accelerator card dedicated computing layer executes the N+1th batch of core computing tasks in parallel, and the CPU general-purpose computing layer completes the parsing, splitting, and resource pre-scheduling of the N+2th batch of tasks in parallel. The three-level architecture operates independently and does not block each other, and ensures the consistency of cross-batch data through a unified timing synchronization mechanism and batch marking mechanism, thereby achieving parallel accelerated processing of multi-batch tasks across the entire chain.

[0029] The multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method provided in this application can overcome the limitations of traditional two-level heterogeneous architectures and in-memory computing separation architectures. By constructing a three-level hierarchical topology structure of a CPU general computing layer, an accelerator card dedicated computing layer, and a storage computing layer, it establishes a closed-loop in-memory computing pipeline that connects the entire "control-acceleration-storage computing" chain. This effectively reduces the frequent cross-level migration of data between memory, video memory, and storage media, and lowers bus transmission latency and bandwidth consumption. Simultaneously, through a dual-mode cooperation of serial connection of the three-level pipeline and staggered batch parallel collaboration, each level can operate independently and without blocking each other when processing different batches of tasks, achieving uninterrupted task streaming processing and significantly improving the overall system throughput performance and computing resource utilization. Furthermore, by sinking lightweight computing capabilities to the storage end, it fully utilizes the idle computing power of the storage layer, avoids resource redundancy and waste, and balances the integrity of single-task processing with the ultra-high throughput requirements of multi-batch parallel processing, significantly enhancing the system's adaptability to complex computing scenarios.

[0030] Optionally, the multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method provided in this application embodiment further includes a scene adaptive dynamic scheduling step: Based on the task decomposition rule base, the business tasks are divided into lightweight general-purpose tasks, computing power accelerated tasks, and high-concurrency tasks according to their functional attributes. Label all tasks with priority tags, resource level tags, and timeliness tags to generate a standardized task distribution list; Automatically switch job modes based on task type and real-time system load status: For lightweight general-purpose tasks, a two-level collaborative mode of CPU and storage computing layers is enabled; for computing power-accelerated tasks, a three-level full-link serial pipeline mode is enabled; and for massive concurrent tasks, a multi-level parallel collaborative mode is activated.

[0031] Optionally, the multi-level heterogeneous storage-computing convergence architecture and pipeline implementation method provided in this application embodiment integrates an embedded field-programmable gate array or a smart solid-state drive in the storage computing layer. The local embedded computing unit is configured to perform data pre-processing and post-processing locally on the storage medium, so that the data is cleaned, normalized and verified in the storage layer before being transmitted outward.

[0032] Optionally, in the multi-level heterogeneous in-memory computing convergence architecture and pipeline implementation method provided in the embodiments of this application, the dedicated computing layer of the accelerator card adopts a multi-accelerator card array cluster deployment, and multiple accelerator cards work together in parallel. The CPU general computing layer is further configured to dynamically adjust the number of accelerator cards participating in the computation and the task allocation ratio according to the data scale and computing power requirements of the task.

[0033] Optionally, the unified timing synchronization mechanism in the multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method provided in this application embodiment includes: Assign a globally unique batch identifier to each batch of tasks; Each level in the three-tier architecture carries the batch identifier when processing tasks, and verifies the consistency of the batch identifier when the task is transmitted.

[0034] Optionally, the multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method provided in this application embodiment include at least one of process scheduling, logical judgment, permission verification, exception handling and result integration in the lightweight general-purpose task; The computing power acceleration tasks include at least one of matrix operations, model reasoning, and high-density floating-point computing. The massive concurrent tasks include at least one of the following: data cleaning, deduplication, format standardization, aggregation statistics, and pre-computation.

[0035] Optionally, the multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method provided in this application embodiment further includes a fault-tolerant processing step: When any level detects a task execution anomaly, the anomaly information is reported to the CPU general computing layer; The CPU general computing layer performs task retry, task migration to a backup computing unit, or downgrade processing according to the exception type, and updates the priority tag and resource level tag in the standardized task distribution list.

[0036] Specifically, this application discloses a multi-level heterogeneous in-memory computing converged architecture, such as... Figure 2As shown, the overall structure adopts a top-down three-level hierarchical heterogeneous topology, consisting of a CPU general computing layer, an accelerator card dedicated computing layer, and a storage computing layer. The three-level architecture is compatible with a dual-mode collaborative operation mechanism, enabling pipelined serial connection operations and multi-dimensional parallel collaborative operations, thus constructing a full-link closed-loop storage-computing pipeline that includes CPU global scheduling and control, accelerator card core computing power acceleration, storage layer preprocessing, and finalized computation.

[0037] The CPU general-purpose computing layer, as the top-level control core, is responsible for global resource management, task splitting, priority scheduling, and instruction distribution. The accelerator card dedicated computing layer, as the mid-level computing power core, is interconnected with the CPU general-purpose computing layer at high speed to undertake intensive parallel computing tasks and accelerate core computing power. The storage computing layer, as the bottom-level in-memory computing core, integrates data storage and embedded lightweight computing capabilities, completing data preprocessing, post-processing, data verification, and local computing and storage tasks. The three layers are connected by a high-speed interconnect bus to form an integrated topology link, which can adaptively switch between serial pipeline operation mode and multi-layer parallel collaborative operation mode according to task characteristics, realizing deep integration of in-memory computing and efficient collaboration across the entire link.

[0038] (1) CPU general computing layer The CPU general-purpose computing layer serves as the system's global control layer, general-purpose computing support layer, and pipeline timing scheduling core. It is responsible for system initialization, unified management and control of hardware resources, global task decomposition and priority scheduling; it performs general logical operations, complex branch judgments, task start / stop control, anomaly monitoring and fault tolerance handling; and dynamically allocates accelerator card computing resources and storage computing layer job resources according to task computing power requirements and data scale, generating unified scheduling instructions for the three-level pipeline to achieve orderly collaborative management between upper and lower levels. It possesses general-purpose programmable characteristics, adaptable to various upper-layer business scenarios, and provides a unified timing benchmark and process scheduling support for the overall in-memory computing pipeline.

[0039] (2) Dedicated computing layer for accelerator cards The dedicated computing layer of the accelerator card serves as the core computing power acceleration layer and the hub for intensive computing processing. It receives structured tasks from the CPU's general-purpose computing layer, focusing on completing high-density, repetitive, and parallel core computing tasks such as matrix operations, model inference, and feature extraction. It responds to upper-layer control commands and provides standardized intermediate computation data to the lower storage and computing layer, completing the core computing power acceleration process. It supports multi-accelerator card array cluster deployment, enabling horizontal scaling of computing power. With strong parallel computing power and outstanding dedicated computing performance, it can compensate for the insufficient parallel computing power of the CPU and handle the core computing power load of the pipeline.

[0040] (3) Storage and computing layer The storage and computing layer serves as the underlying support layer for the system's integrated storage and computing capabilities, as well as the data quality processing and localization computing layer. It integrates high-speed storage media and embedded computing units, breaking through the single function of traditional storage systems that only provide read and write operations. Building upon high-speed data read / write and persistent storage, it performs data processing tasks such as data cleaning, fragmentation and normalization, format conversion, result verification, and lightweight aggregation. It directly interfaces with the dedicated computing layer of the accelerator card, enabling localized computing and storage, reducing frequent data movement across layers. It possesses integrated storage and computing capabilities and localized data operation capabilities, effectively reducing bus transmission overhead and compensating for the technical shortcomings of traditional storage layer architectures that lack computing power.

[0041] Figure 3 shows the overall workflow provided in the embodiment of this application. This workflow is based on a three-level heterogeneous architecture of CPU main control layer + accelerator card acceleration layer + storage computing layer, which is adapted to high concurrency, high density and uninterrupted pipeline operation scenarios.

[0042] (1) Implementation process of serial operation on the assembly line S1, the CPU general computing layer receives business tasks from the upper layer, completes task parsing, granular decomposition and priority sorting, and generates hierarchical execution instructions adapted to the three-level architecture. S2 and the CPU general computing layer will autonomously execute complex control tasks and distribute parallel computing tasks to the accelerator card dedicated computing layer and data quality processing tasks to the storage computing layer. S3, the dedicated computing layer of the accelerator card initializes the computing power unit, performs high-density parallel computing, and generates standardized intermediate data; S4. Intermediate data is pushed to the storage and computing layer, where the storage and computing layer completes data processing tasks such as data cleaning, format optimization and consistency verification through local embedded computing units. S5, the storage and computing layer completes data regularization and persistent storage; S6, the dedicated computing layer of the accelerator card and the storage computing layer will send the processed results back to the general computing layer of the CPU. S7 and CPU general computing layer complete the final result aggregation and business output, completing a single pipeline closed-loop operation; The S8 and CPU general computing layers update the pipeline status and dynamically schedule the next round of pipeline tasks; each layer continuously iterates and executes the above process to achieve uninterrupted serial processing of tasks.

[0043] (2) Implementation process of multi-level parallel collaborative operation Building upon the aforementioned serial pipeline, the system initiates a batch-staggered parallel mechanism: while the storage computing layer processes the Nth batch of data regularization and write-in tasks, the dedicated computing layer of the accelerator card executes the N+1th batch of core computing tasks in parallel, and the CPU general-purpose computing layer completes the parsing, splitting, and resource pre-scheduling of the N+2th batch of tasks in parallel. This three-tiered architecture operates independently without blocking each other, ensuring data consistency through a unified timing synchronization mechanism and batch marking mechanism, thus achieving parallel accelerated processing of multiple batches of tasks across the entire chain.

[0044] (3) Dynamic scheduling and adaptation optimization Task scheduling is optimized based on task attributes: Based on a task decomposition rule base, complex business tasks are categorized into three minimum atomic task units according to their functional attributes: lightweight general-purpose tasks, including process scheduling, logical judgment, permission verification, exception handling, and result integration; these tasks are highly logical and cannot be executed in parallel. Computationally accelerated tasks, including matrix operations, model inference, and high-density floating-point calculations; these tasks have high computational requirements but can be executed in parallel. High-concurrency tasks, including data cleaning, deduplication, format standardization, aggregation statistics, and pre-computation; these tasks are high I / O data-intensive, and memory access bandwidth and speed significantly impact their efficiency. Based on business priority, urgency, and resource usage registration, all tasks are tagged with priority labels, resource level labels, and timeliness labels, generating a standardized task distribution list.

[0045] The system achieves adaptive scheduling based on real-time load and task attributes: for lightweight general-purpose tasks, it enables a two-level collaborative mode of CPU and storage computing layers to save accelerator card resources; for computing power acceleration tasks, it enables a three-level full-link serial pipeline mode to ensure computing accuracy and process stability; for massive concurrent tasks, it activates a multi-level parallel collaborative mode to maximize system throughput performance and achieve scenario-based intelligent adaptation.

[0046] This application pioneers a three-tier heterogeneous in-memory computing converged topology, breaking through the technical limitations of traditional two-tier heterogeneous architectures and in-memory computing separation architectures. It achieves deep integration of storage and computing functions, fully leveraging lightweight computing resources on the storage side and enriching the hierarchical collaboration dimensions of heterogeneous computing architectures. Through a dual-mode combination of three-tier pipeline serial connection and multi-tier parallel collaboration, it achieves uninterrupted streaming operations across the entire link, effectively reducing redundant data handling, lowering system transmission latency, and significantly improving computing power utilization and task concurrency throughput. The three-tiered functional division is clear and each performs its specific function: the CPU focuses on global control and task scheduling, the accelerator card handles the core's high-density computing load, and the storage layer completes localized in-memory computing preprocessing tasks, achieving fine-grained on-demand resource allocation and eliminating idle computing power and resource redundancy waste. It is compatible with both serial high-precision operation and parallel high-throughput operation modes, flexibly adapting to various business scenarios such as big data processing, artificial intelligence inference, high-performance computing, and edge computing scheduling, exhibiting excellent architectural scalability and hardware compatibility. By constructing a complete integrated storage and computing pipeline that combines control, acceleration, and storage, the shortcomings of traditional architectures in terms of broken end-to-end operations are overcome, forming a closed-loop collaborative operation system from task scheduling and computing power to data landing.

[0047] This application also provides a multi-level heterogeneous in-memory computing converged architecture and pipeline implementation device, including: The service receiving module is used to receive service tasks through the CPU general computing layer, parse and split them into hierarchical execution instructions; The task distribution module is used to distribute parallel computing tasks to the accelerator card dedicated computing layer through the CPU general computing layer, and to distribute data quality processing tasks to the storage computing layer. The core computing module is used to perform core computing through the dedicated computing layer of the accelerator card, generate standardized intermediate data, and push it to the storage computing layer; The data processing module is used to clean, optimize the format, and verify the consistency of the standardized intermediate data through the storage computing layer using a local embedded computing unit, thereby completing data regularization and persistent storage. The result output module is used to aggregate and output the processing results through the CPU general computing layer to form a single pipeline closed loop. The state update module is used to update the pipeline state through the CPU general computing layer and dynamically schedule the next round of tasks to achieve streaming serial processing. The parallel computing module is used to simultaneously execute the next batch of core computing tasks in parallel by the accelerator card dedicated computing layer while the storage computing layer processes the current batch of data regularization and landing tasks, and the CPU general computing layer completes the parsing and resource pre-scheduling of the next batch of tasks in parallel.

[0048] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0049] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0050] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0051] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0052] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0053] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0054] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0055] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0056] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

[0057] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0058] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method, characterized in that, Includes the following steps: The CPU general computing layer receives business tasks, parses and breaks them down into hierarchical execution instructions; The CPU general computing layer distributes parallel computing tasks to the accelerator card dedicated computing layer, and the data quality processing tasks are distributed to the storage computing layer. Core computations are performed through the dedicated computing layer of the accelerator card, generating standardized intermediate data and pushing it to the storage computing layer; The storage computing layer utilizes local embedded computing units to clean, optimize the format, and verify the consistency of the standardized intermediate data, thereby completing data regularization and persistent storage. The CPU general computing layer aggregates and outputs the processing results to form a single pipeline closed loop. The pipeline state is updated through the general computing layer of the CPU, and the next round of tasks is dynamically scheduled to achieve streaming serial processing. While the storage computing layer processes the current batch of data organization and landing tasks, the accelerator card dedicated computing layer executes the next batch of core computing tasks in parallel, and the CPU general computing layer completes the parsing and resource pre-scheduling of the next batch of tasks in parallel.

2. The multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method according to claim 1, characterized in that, It also includes a scene-adaptive dynamic scheduling step: Based on the task decomposition rule base, the business tasks are divided into lightweight general-purpose tasks, computing power accelerated tasks, and high-concurrency tasks according to their functional attributes. Label all tasks with priority tags, resource level tags, and timeliness tags to generate a standardized task distribution list; Automatically switch job modes based on task type and real-time system load status: For lightweight general-purpose tasks, a two-level collaborative mode of CPU and storage computing layers is enabled; for computing power-accelerated tasks, a three-level full-link serial pipeline mode is enabled; and for massive concurrent tasks, a multi-level parallel collaborative mode is activated.

3. The multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method according to claim 1, characterized in that, The storage computing layer integrates an embedded field-programmable gate array or a smart solid-state drive. The local embedded computing unit is configured to perform data preprocessing and postprocessing locally on the storage medium, so that the data is cleaned, normalized and verified in the storage layer before being transmitted outward.

4. The multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method according to claim 1, characterized in that, The dedicated computing layer for the accelerator cards is deployed in a multi-accelerator card array cluster, with multiple accelerator cards working together in parallel. The CPU general computing layer is further configured to dynamically adjust the number of accelerator cards participating in the computation and the task allocation ratio according to the data scale and computing power requirements of the task.

5. The multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method according to claim 1, characterized in that, The unified timing synchronization mechanism includes: Assign a globally unique batch identifier to each batch of tasks; Each level in the three-tier architecture carries the batch identifier when processing tasks, and verifies the consistency of the batch identifier when the task is transmitted.

6. The multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method according to claim 2, characterized in that, The lightweight general-purpose task includes at least one of process scheduling, logical judgment, permission verification, exception handling and result integration; The computing power acceleration tasks include at least one of matrix operations, model reasoning, and high-density floating-point computing. The massive concurrent tasks include at least one of the following: data cleaning, deduplication, format standardization, aggregation statistics, and pre-computation.

7. The multi-level heterogeneous in-memory computing converged architecture and pipeline implementation method according to claim 1, characterized in that, It also includes fault tolerance processing steps: When any level detects a task execution anomaly, the anomaly information is reported to the CPU general computing layer; The CPU general computing layer performs task retry, task migration to a backup computing unit, or downgrade processing according to the exception type, and updates the priority tag and resource level tag in the standardized task distribution list.

8. A multi-level heterogeneous in-memory computing converged architecture and pipeline implementation device, characterized in that, include: The service receiving module is used to receive service tasks through the CPU general computing layer, parse and split them into hierarchical execution instructions; The task distribution module is used to distribute parallel computing tasks to the accelerator card dedicated computing layer through the CPU general computing layer, and to distribute data quality processing tasks to the storage computing layer. The core computing module is used to perform core computing through the dedicated computing layer of the accelerator card, generate standardized intermediate data, and push it to the storage computing layer; The data processing module is used to clean, optimize the format, and verify the consistency of the standardized intermediate data through the storage computing layer using a local embedded computing unit, thereby completing data regularization and persistent storage. The result output module is used to aggregate and output the processing results through the CPU general computing layer to form a single pipeline closed loop. The state update module is used to update the pipeline state through the CPU general computing layer and dynamically schedule the next round of tasks to achieve streaming serial processing. The parallel computing module is used to simultaneously execute the next batch of core computing tasks in parallel by the accelerator card dedicated computing layer while the storage computing layer processes the current batch of data regularization and landing tasks, and the CPU general computing layer completes the parsing and resource pre-scheduling of the next batch of tasks in parallel.

9. A multi-level heterogeneous in-memory computing converged architecture and pipeline implementation device, characterized in that, The method includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program that, when executed by the processing unit, causes the processing unit to perform the steps of the method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, It stores a computer program that can be executed by a multi-level heterogeneous in-memory computing architecture and pipeline implementation device. When the computer program is run on the multi-level heterogeneous in-memory computing architecture and pipeline implementation device, the multi-level heterogeneous in-memory computing architecture and pipeline implementation device performs the steps of the method according to any one of claims 1 to 7.

Images