High-efficiency synergic artificial intelligence multi-core computing device and method

Abstract

The application belongs to the technical field of artificial intelligence computing, and particularly relates to a high-efficiency cooperative artificial intelligence multi-core computing device and method. A global task preprocessing module of the application can collect a network-on-chip congestion vector based on a real-time load sensing module, dynamically evaluate communication cost and computing benefit, and accordingly adjust operator splitting granularity in real time. When the network is congested, operators are combined to increase granularity and reduce communication frequency and data volume. When the network is idle, operators are split to improve parallelism. This adaptive granularity adjustment mechanism enables the task parallel strategy to cooperate with the real-time network state, ensures that the benefit brought by parallel computing is always higher than the communication overhead, thereby optimizing the utilization efficiency of on-chip communication resources and improving the stability and performance of the system under heavy load.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence computing technology, specifically relating to a high-efficiency collaborative artificial intelligence multi-core computing device and method. Background Technology

[0002] Currently, high-performance heterogeneous multi-core system-on-a-chip (SoC) for artificial intelligence computing has become mainstream. These systems typically integrate computing cores with multiple architectures (such as general-purpose CPUs, vector DSPs, tensor NPUs, etc.) and interconnect them through on-chip networks to handle complex computing tasks such as large model inference and multimodal generation.

[0003] Existing task scheduling technologies mainly focus on load balancing of the computing core itself, that is, to distribute computing tasks evenly to various idle or lightly loaded computing cores. However, this "core-centric" scheduling strategy ignores the "roads" that computing tasks rely on when flowing between many computing cores—that is, the communication status of the on-chip network. When a large number of fine-grained tasks exchange and synchronize data on the on-chip network at the same time, huge communication traffic will be generated, which can easily lead to network congestion. At this time, the time for tasks to wait for data communication to complete may far exceed their own computing time. Summary of the Invention

[0004] To address the aforementioned shortcomings in the existing technology, this invention provides a highly efficient collaborative artificial intelligence multi-core computing device and method to solve the problems mentioned in the background technology.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A high-efficiency collaborative artificial intelligence multi-core computing device, comprising: A heterogeneous multi-core computing cluster consists of multiple computing cores with different architectures deployed on different physical clusters on a chip; The on-chip interconnect network module connects the various modules and computing cores of the heterogeneous multi-core computing cluster for data interaction and instruction transmission; The global task preprocessing module receives artificial intelligence computing tasks, parses the tasks based on the intermediate representation of the computation graph and the real-time execution status, and labels the operators with attribute tags based on the computation characteristics, parallelism and memory access characteristics of the operators; and dynamically adjusts the splitting granularity of the operators based on the real-time congestion status of the on-chip interconnect network to generate atomic computing tasks with attribute tags, or reorganizes multiple atomic computing tasks with data dependency strength exceeding a preset threshold into computing cluster tasks with a forced same-cluster scheduling flag. The distributed core-level scheduling agent cluster includes a core-level scheduling agent corresponding to each computing core. It is used to collect the real-time status of the corresponding computing core, predict the task completion time based on the current task execution progress, and send a pre-request request carrying the computing core status and the predicted completion time to the cross-core collaborative arbitration module before the task is completed. The cross-core collaborative arbitration module is used to perform dynamic arbitration, matching, and pre-distribution of tasks based on the attribute tags of the atomic computing tasks or computing cluster tasks and the pre-claim requests of each core-level scheduling agent. The real-time load awareness module is used to periodically collect load data from all computing cores and congestion data from each link of the on-chip interconnect network, and to build and update the overall system status view.

[0006] Preferably, the global task preprocessing module includes: A dynamic fragmentation adjustment unit is used to dynamically adjust the operator splitting granularity based on the on-chip interconnection network congestion vector by evaluating communication costs and computational benefits. The communication cost is calculated based on the weighted sum of link delay and bandwidth utilization, and the computational benefits are quantified based on the parallel execution efficiency of the operators. When the ratio of communication cost to computational benefits is greater than a first threshold, operators are merged to increase granularity; when the ratio is less than or equal to the first threshold and network congestion is lower than a second threshold, operators are split to decrease granularity. The computation cluster reorganization unit is used to reorganize the multiple atomic computation tasks into a computation cluster task when the data dependency strength between multiple atomic computation tasks exceeds a third threshold, and to mark the computation cluster task with a forced same-cluster scheduling identifier.

[0007] Preferably, each of the core-level scheduling agents includes: The shadow scheduling unit is used to predict task completion time based on the current task execution progress using a linear regression model; The task pre-claim unit is used to send the pre-claim request to the cross-core collaborative arbitration module within a preset time window before the predicted task completion time.

[0008] Preferably, the cross-core collaborative arbitration module includes: The topology-aware adaptation arbitration unit is used to match the best computing core for the task. For computing cluster tasks, it prioritizes matching idle computing core groups within the same physical cluster. For atomic computing tasks, it selects the computing core with a load below the idle threshold and the closest physical distance as candidates based on the adaptation core type in its attribute label, and uses a weighted multidimensional scoring algorithm to determine the target scheduling core from the candidate cores. The task pre-distribution unit is used to pre-distribute the new tasks that have been arbitrated and matched, along with their corresponding data indexes, to the core-level scheduling agent of the target computing core before the current task of the target computing core is completed, so as to trigger the data prefetching operation.

[0009] Preferably, the cross-core collaborative arbitration module further includes: The precision-driven flexible scheduling unit is used to schedule atomic computing tasks to non-adaptive cores for execution when there are no available computing cores for both the optimal and suboptimal adapted core types. The precision compensation management unit is used to collect the result deviation data of tasks executed on non-adapted cores, use the least squares method to fit and generate precision compensation parameters based on historical deviation data, and send precision compensation instructions to the core-level scheduling agent.

[0010] Preferably, the real-time load sensing module includes: The threshold warning unit is used to generate a warning signal and send it to the cross-core collaborative arbitration module and the global task preprocessing module when the load rate of any computing core exceeds the overload threshold or the congestion level of any network link exceeds the congestion threshold, so as to trigger task migration or task splitting granularity adjustment.

[0011] A high-efficiency collaborative artificial intelligence multi-core computing method, applied to a high-efficiency collaborative artificial intelligence multi-core computing device according to any one of claims 1-6, the method comprising: S1. Dynamic task preprocessing: Parse the input computation task, label the operators with attribute tags, and dynamically adjust the operator splitting granularity based on the real-time network status to generate atomic computation tasks, or reorganize strongly data-dependent atomic computation tasks into computation cluster tasks. S2. Distributed pre-request: The scheduling agent of each computing core monitors the status of its own core in real time and predicts the completion time of the current task. Before the task is completed, it sends a pre-request request to the cross-core collaborative arbitration module. S3. Topology-aware arbitration and pre-distribution: The cross-core collaborative arbitration module matches the best computing core for a task based on task attributes, pre-request requests, overall system load, and physical topology, and pre-distributes the new task and data index before the current task on that core ends. S4. Flexible scheduling and accuracy compensation: When no suitable core is available, the task is dynamically quantified and downgraded to be scheduled to be executed on a non-suitable core, and the execution deviation is collected. Based on the historical deviation, the compensation parameters are fitted to perform closed-loop accuracy compensation for subsequent similar tasks. S5. Task Execution and Status Update: The computing core executes the assigned tasks, reports the status upon completion, triggers a new round of pre-claiming, and updates the load view of the entire system.

[0012] Preferably, in step S1, the dynamic adjustment of operator splitting granularity includes: evaluating communication costs based on on-chip network congestion vectors, evaluating computational benefits based on operator parallel potential, and determining whether to merge or split operators based on the ratio of communication costs to computational benefits.

[0013] Preferably, in step S3, matching the best computing core for the atomic computing task includes: firstly, screening computing cores with a load below the idle threshold among the optimal matching core types; if there are multiple, selecting the computing core with the lightest load and the closest physical distance to the task data dependent core.

[0014] Preferably, in step S4, the closed-loop accuracy compensation includes: establishing a deviation mapping table with operator type and execution kernel type as keys, using the least squares method to fit the deviation pattern of the operator executed on a specific kernel, obtaining the weight compensation coefficient and the bias adjustment amount, and generating a compensation instruction accordingly, and performing real-time correction in subsequent execution according to "compensated result = original result × weight compensation coefficient + bias adjustment amount".

[0015] Compared with the prior art, the present invention has the following advantages: 1. The global task preprocessing module of this invention can dynamically evaluate the communication cost and computational benefit based on the on-chip network congestion vector collected by the load real-time perception module, and adjust the operator splitting granularity in real time accordingly. When the network is congested, operators are merged to increase granularity and reduce communication frequency and data volume; when the network is idle, operators are split to improve parallelism. This adaptive granularity adjustment mechanism enables the task parallel strategy to coordinate with the real-time network status, ensuring that the benefits of parallel computing are always higher than its communication overhead, thereby optimizing the utilization efficiency of on-chip communication resources and improving the stability and performance of the system under heavy load. 2. This invention enables computing cores to actively and distributedly claim tasks through a distributed core-level scheduling agent cluster, and combines it with a cross-core collaborative arbitration module for dynamic matching based on global state. This "request-arbitration" distributed collaborative mechanism allows the system to respond in real time to the load changes of each computing core, dynamically and flexibly scheduling tasks to the most suitable idle or lightly loaded computing cores, achieving fine-grained load balancing, thereby maximizing the mining and utilization of the computing power potential of all heterogeneous computing units on the chip and improving the overall throughput of the system. 3. This invention effectively eliminates task switching and idle waiting time of computing cores by introducing a collaborative pipeline mechanism of "prediction-pre-request-pre-distribution". Specifically, the distributed core-level scheduling agent predicts the completion time of the current task based on a linear regression model and issues a pre-request request in advance; the cross-core collaborative arbitration module completes arbitration matching in advance accordingly, and the task pre-distribution unit distributes the new task and data index in advance before the current task of the target computing core is completed. This mechanism ensures that the computing core can start executing the new task with almost "zero waiting" after completing the current task, and at the same time triggers data prefetching, realizing deep overlap of "computation-communication-scheduling", thereby significantly reducing the overall execution latency of complex task flows.

[0016] 4. This invention, through a topology-aware adaptation arbitration unit, prioritizes the physical topology distance between computing cores during task matching and prioritizes the allocation of computing core groups within the same physical cluster to computing cluster tasks with mandatory same-cluster identifiers. This topology-aware scheduling strategy maximizes data locality, transforming long-distance, high-latency on-chip network communication into high-speed intra-cluster bus or neighbor communication, significantly reducing synchronization latency caused by data dependencies. Furthermore, when resources are scarce, the precision-driven flexible scheduling unit and the precision compensation management unit work together to allow tasks to be dynamically quantized and scheduled to non-adapted cores for execution. The calculation results are corrected in real time through closed-loop compensation parameters fitted based on historical deviation data. This flexible degradation and closed-loop compensation mechanism enhances the robustness and reliability of the system during peak load periods. Attached Figure Description

[0017] Figure 1 This is a framework diagram of a high-efficiency collaborative artificial intelligence multi-core computing device according to the present invention; Figure 2 This is a flowchart of an efficient collaborative artificial intelligence multi-core computing method for invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present invention, the technical solution of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0019] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual images. They should not be construed as limiting the scope of this patent. To better illustrate the embodiments of the present invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0020] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "inner," and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present patent. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0021] In the description of this invention, unless otherwise explicitly specified and limited, the term "connection" or similar designation indicating a connection between components should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral part; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can refer to the internal communication between two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0022] Example 1: like Figure 1-2 The diagram illustrates a high-efficiency collaborative AI multi-core computing device and method. In one embodiment, the heterogeneous multi-core computing cluster includes at least two types of cores: general-purpose computing cores, tensor acceleration cores, vector computing cores, and dedicated acceleration cores. The specific configuration can be flexibly tailored to the actual application scenario. For example, in a deep learning inference scenario, four general-purpose computing cores, eight tensor acceleration cores, and four dedicated acceleration cores can be configured. The heterogeneous multi-core computing cluster is divided into several physical clusters based on their physical location. The physical cluster division rule is as follows: two to four computing cores that are physically adjacent on the chip and share the same local high-speed bus are grouped into one physical cluster. Inter-cluster interconnection is achieved using a 2DMesh topology on-chip network, with XY-order routing as the routing algorithm. Communication between intra-cluster computing cores and inter-cluster communication is achieved through hardware bus arbitrators and on-chip network VLAN hardware isolation to avoid bus contention and data interference. The intra-cluster local high-speed bus adopts the AXI4-Stream protocol, with a data width of 256 bits and a working frequency of 1.2 GHz, consistent with the computing core frequency, ensuring low-latency data interaction within the cluster.

[0023] The general-purpose computing core adopts a RISC-V architecture with a main frequency of 1.2GHz, used to execute control flow, scalar computation, and irregular computation tasks, ensuring the normal operation of the entire device's control logic. The tensor acceleration core adopts a systolic array architecture, supporting FP16 / FP32 precision computation, used to execute matrix multiplication and convolution-type tensor parallel computation tasks. The vector computation core supports 256-bit vector operations, used to execute vector operations and data preprocessing tasks. The dedicated acceleration core adopts a customized architecture, specifically designed to execute Transformer operators, quantization inference, and activation function-type dedicated operator tasks. The dedicated acceleration core adopts a customized hard-wired architecture, integrating a multi-head attention computation array, a SoftMAX hardware computing unit, and a dedicated quantization inference multiplier array, specifically designed for executing... The Transformer operator, quantized inference, and activation function-based dedicated operator tasks are all configured with corresponding core-level scheduling agents and on-chip network access ports for each computing core. The access ports adopt the AXI4 bus protocol, support AXI burst transmission mode, and the burst transmission length is configured to 16 / 32 / 64 bytes. The transmission timing follows the 1.2GHz on-chip network synchronous clock protocol, and data transmission is completed in a 4-stage pipeline. Computing cores within the same physical cluster share a high-speed local bus, effectively improving the collaborative computing efficiency and data interaction speed of computing cores within the same cluster, and reducing data transmission latency within the same cluster. The physical distance quantization rule is predefined: physical distance in Mesh topology = horizontal hop count between cores + vertical hop count. The fewer the hop count, the closer the physical location, providing a unified quantization standard for subsequent topology-aware scheduling. The global task preprocessing module includes a computation graph analysis unit, an operator classification and labeling unit, a dynamic piecewise adjustment unit, and a computation cluster reorganization unit. These units work together to complete the pre-optimization of the task.

[0024] The computation graph parsing unit employs a combination of static parsing and dynamic tracing. Static parsing occurs during the initial task loading or compilation phase, using the ONNX / TensorRT intermediate representation parsing tool to parse the intermediate representation of the computation graph exported from the TensorFlow and PyTorch frameworks layer by layer, resulting in a directed acyclic graph. Nodes are parsed as atomic operators (such as convolution Conv, matrix multiplication MatMul, and activation function ReLU), while edges represent the flow of tensor data. This process extracts two key dependencies: data dependencies (determining producer-consumer relationships between operators, i.e., the output tensor of one operator is the input of another) and control flow dependencies (identifying conditional branches, loops, and other structures in the model, clarifying the triggering conditions of operators under different execution paths). Dynamic tracking is performed in real time during task execution. It receives operator execution status and data transmission results from the execution monitoring and deviation acquisition unit, updates the initial computation graph node by node according to the execution order, deletes completed nodes, adds dynamically generated operator nodes, and continuously updates the initial computation graph generated by static analysis. This provides a direct basis for runtime optimization by the dynamic sharding adjustment unit and the computation cluster reorganization unit, ensuring the accuracy of the analysis results and clearly outlining the computational logic and dependencies of the task. The operator classification and labeling unit labels each operator node with corresponding operator attribute tags based on the operator's computational characteristics, parallelism, and memory access characteristics. These tags are stored using 32-bit binary encoding and specifically include operator type (8 bits), optimal adapted kernel type (4 bits), suboptimal adapted kernel type (4 bits), computational cost (4 bits), memory access cost (4 bits), parallelizability (2 bits), data dependency strength (2 bits), and precision sensitivity (2 bits), with 2 bits reserved for future use. The encoding mapping rule is as follows: 4-bit encoding for optimal / suboptimal adapted kernel type. : 0001 = General-purpose computing core, 0010 = Tensor acceleration core, 0100 = Vector computing core, 1000 = Dedicated acceleration core; 4-bit encoding of computational load / memory access: 0001 = Very small, 0010 = Small, 0100 = Medium, 1000 = Large; 2-bit encoding of parallelizability: 01 = Low, 10 = Medium, 11 = High; 2-bit encoding of data dependency strength: 01 = Weak, 10 = Medium, 11 = Strong; 2-bit encoding of precision sensitivity: 01 = Low, 10 = Medium, 11 = High, providing accurate basis for subsequent task matching and scheduling; The dynamic fragmentation adjustment unit is used to evaluate communication costs and computational benefits based on the on-chip interconnection network congestion vector using a preset quantization model, and dynamically adjusts the operator splitting granularity based on the trade-off between the two. The on-chip interconnection network congestion vector is generated by the on-chip network congestion acquisition unit, which collects bandwidth utilization, forwarding delay, and buffer backlog data of each link, and then performs linear normalization. The linear normalization formula is: Normalized value = (Actual value - Minimum value) / (Maximum value - Minimum value). The minimum and maximum values ​​are predetermined or dynamically obtained based on the on-chip network design specifications (such as theoretical limits or the actual range in the previous sampling period). The vector dimension is consistent with the number of on-chip links, and each element corresponds to the congestion quantization value of a single link, ranging from 0 to 1. The communication cost is calculated using a weighted summation model, with the specific formula as follows: Where α and β are preset weighting coefficients ( Link latency and bandwidth utilization are converted into dimensionless values ​​of 0 to 1, and the communication cost is ultimately output as a quantized value of 0 to 1. The computational benefit is quantized based on the operator parallel execution efficiency, which is defined as the ratio of multi-core computing power utilization to single-core computing power utilization after operator splitting (i.e., speedup ratio). The computational benefit = operator parallel execution efficiency - 1. The computational benefit is a non-negative quantized value, and the larger the value, the higher the parallel gain. When the computational benefit = 0, the default cost-benefit ratio = 2, which forces operator merging to avoid division by zero anomalies. The dynamic sharding adjustment unit further calculates the cost-benefit ratio = communication cost / computational benefit, and combines it with preset thresholds to determine the granularity of the task sharding operator (all thresholds are calibrated based on standard workloads using the SystemC and VitisHLS simulation platforms). When the predicted communication latency is too high (i.e., the inter-core communication latency exceeds the preset ratio of 30% of the computation time) and the cost-benefit ratio is >1, the operator is merged to increase the granularity and reduce the frequency of inter-core communication. When the network is idle (i.e., the on-chip interconnection network congestion is lower than the preset threshold of 50% bandwidth utilization) and the cost-benefit ratio is ≤1, the operator is split to reduce the granularity, improve parallel efficiency, and achieve dynamic adaptation of task sharding granularity with on-chip communication status and computational gain. The formula for calculating data dependency strength in the computation cluster reorganization unit is: Data dependency strength = Number of directly dependent operators / Total number of operators. When the calculated value exceeds a preset threshold (marked as 0.8 in this embodiment), reorganization is triggered, and multiple atomic computation tasks with data dependency strength exceeding the threshold are reorganized into computation cluster tasks and marked with a forced same-cluster scheduling flag. The identifier is represented by a 1-bit binary number and sent to the cross-core collaborative arbitration module to avoid high communication overhead caused by cross-cluster scheduling for tasks with strong data dependencies and reduce task execution latency. Each core-level scheduling agent in the distributed core-level scheduling agent cluster includes a status acquisition unit, a shadow scheduling unit, a task pre-request unit, a local task scheduling unit, and an execution monitoring and deviation acquisition unit. The status acquisition unit adopts a real-time sampling method, and the sampling frequency is consistent with the acquisition cycle of the load real-time perception module (i.e., 12 system clock cycles). It collects the computing power utilization, task queue length, memory access bandwidth utilization, and power consumption data of the corresponding computing core in real time. The accuracy of computing power utilization is 1%, and the accuracy of power consumption data acquisition is 0.1W. After collection, it is reported to the load real-time perception module through the on-chip high-speed interface to ensure the real-time performance and accuracy of load data. A univariate linear regression model is used to predict task completion time. The model formula is as follows: ,in The execution time is [time already elapsed]. To predict the total time, The slope of computing power utilization rate, The time consumed by the operator base; The coefficients are calibrated by fitting data from 100 historical executions of the same type of operator, with the required fitting accuracy. If the prediction deviation exceeds 10%, it will automatically switch to historical average prediction to avoid abnormal prediction; the task execution trajectory curve is generated by sampling at the system clock cycle interval and recording the executed time and remaining time point by point. It is used to predict the task completion time based on the current task execution progress, operator calculation amount and corresponding computing core computing power parameters, using a linear regression algorithm to generate the task execution trajectory curve, providing accurate time basis for task pre-application. This is used to send a pre-request request carrying the computing core status and predicted completion time to the cross-core collaborative arbitration module within a preset time (approximately 15 system clock cycles) before the task completion. The request data packet has a fixed 64-byte frame structure, containing, in sequence: core type identifier (1B), physical cluster identifier (1B), computing power utilization (1B), predicted completion time (4B), checksum (1B), and reserved bits (56B), using CRC8 checksum. The data packet is transmitted in data packet format to achieve early task request and seamless connection, avoiding computing core idleness. The local task scheduling unit adopts a non-preemptive priority scheduling algorithm, where high-priority tasks are directly inserted at the head of the queue, and medium / low-priority tasks are... The calculation is sorted by FIFO. The precision compensation formula is: Compensated result = Original result x Weight compensation coefficient + Bias adjustment amount. The compensation precision is not less than 0.001. It is used to receive atomic computing tasks or computing cluster tasks allocated by the cross-core collaborative arbitration module, and schedule them to the corresponding computing cores for execution according to task priority (high priority is real-time inference tasks, medium priority is training tasks, and low priority is auxiliary tasks). The scheduling delay does not exceed 10 system clock cycles. At the same time, it receives precision compensation instructions issued by the cross-core collaborative arbitration module. In the subsequent operator execution process, the precision compensation is completed by adjusting the bias and weight parameters of the operator. The compensation precision is not less than 0.001, ensuring the accuracy of the calculation results. The execution monitoring and deviation acquisition unit uses the operator execution completion node as the breakpoint. The breakpoint is triggered when the operator output tensor is written to the cache. The result deviation is calculated using relative error, with the formula: Deviation = |Ideal Calculated Value - Actual Calculated Value| / Ideal Calculated Value, with a precision of 0.001. The deviation data is stored as FP32 floating-point numbers. The execution progress of the task is monitored in real time using a breakpoint detection method. When the task is completed, the completion status is immediately reported to the cross-core collaborative arbitration module, with a feedback delay of 3 system clock cycles. At the same time, the result deviation data of the non-optimally adapted core execution operator is collected, reported to the cross-core collaborative arbitration module, and a new round of task pre-claim requests is triggered, forming a closed-loop process of task execution and claim. The cross-core collaborative arbitration module includes a task queue management unit, a pre-request processing unit, a topology-aware adaptation arbitration unit, a task pre-distribution unit, a precision-driven flexible scheduling unit, and a precision compensation management unit, and is the scheduling core of the entire device. The task queue management unit receives atomic computing tasks and computing cluster tasks issued by the global task preprocessing module, and constructs a global ready task queue according to task priority and dependency. The queue adopts a first-in-first-out (FIFO) mechanism, and only tasks without prerequisite dependencies are included in the ready queue. The queue capacity is set to 64 tasks. The queue overflow handling rules are as follows: when the queue is full, the earliest low-priority auxiliary task is first eliminated. When there are no low-priority tasks, the task is temporarily stored in the on-chip shared SRAM cache to ensure the orderliness of task execution. The on-chip shared SRAM cache is a shared storage area set in the heterogeneous multi-core computing cluster. This shared storage area refers to a public storage area that can be accessed by multiple computing cores and control modules in the system through the on-chip interconnect network. The pre-request processing unit receives pre-request requests from each core-level scheduling agent. Through load threshold detection (40% idle threshold for general cores, 35% idle threshold for tensor / dedicated acceleration cores), it filters out valid pre-request requests whose corresponding computing core load rate is below the preset idle threshold and which have the capability to execute tasks, while eliminating invalid requests. This improves scheduling efficiency, with request processing latency completed within several system clock cycles. The multi-pre-request conflict handling rules are as follows: when multiple cores simultaneously request the same task, they are sorted by core ID hash value + task priority arbitration to ensure unique task allocation and avoid duplicate execution.

[0025] The physical distance quantization rule in the topology-aware adaptation arbitration unit follows the Mesh topology hop count arithmetic sum rule; a greedy matching algorithm based on weighted multidimensional scoring is adopted. 1. Based on task tags, filter out all computing cores of "optimal fit core type" with a load below the idle threshold to form a candidate core set. If none are found, then progressively filter for candidate cores of "second-best fit core type"; 2. For each computational kernel i in the candidate kernel set, calculate its fitness score according to the formula. ,in, Real-time computing power utilization rate of the core (normalized to 0~1). The length of the task queue. This refers to the physical hop count between the core and the task data-dependent core (or cluster center). To preset weights, 3. Select The highest-scoring computing core is selected as the target scheduling core. If scores are the same, the core with the smaller physical cluster ID is prioritized. For computing cluster tasks, idle computing core groups within the same physical cluster are prioritized. If there are no idle computing core groups within the same physical cluster, the task is rejected and the cluster waits for idle computing cores. The timeout threshold is 100μs. If the timeout occurs, the task is automatically split into atomic tasks, and the forced same-cluster identifier is canceled. The computing core with the lightest load and closest physical distance among the optimal matching core types is prioritized. Among computing cores of the same type, they are sorted according to the priority of load rate from low to high, task queue length from short to long, and physical distance from adjacent data-dependent cores. The optimal target computing core is matched using a greedy algorithm. When there are no valid pre-claim requests for the optimal matching core type, valid pre-claim requests for the second-best matching core type are matched in turn to achieve optimal matching between tasks and computing cores.

[0026] The task pre-distribution unit is used to pre-distribute matched tasks and data indexes before the current task of the target computing core is completed, triggering memory access prefetching to achieve overlap between computation and memory access. Specifically, within about 15 system clock cycles before the current task of the target computing core is completed, the tasks that have completed arbitration matching are distributed to the core-level scheduling agent of the corresponding target computing core. The instruction stream and data index corresponding to the task are distributed synchronously. The instruction stream adopts parallel transmission mode, and the data index points to the on-chip cache address, triggering the memory access prefetching operation of the target computing core. The prefetch advance is 5 clock cycles, which effectively improves the overall computing efficiency. The task pre-distribution unit is also equipped with a fault-tolerant timer. If no "task completion" feedback is received from the target core within two system clock cycles after the target core predicts the completion time, it is determined to be a prediction deviation or execution abnormality. At this time, the unit will send an interrupt signal to the cross-core collaborative arbitration module to cancel the association of the pre-distributed task and put it back into the head of the global ready task queue to wait for the next scheduling arbitration.

[0027] Using a symmetric quantization algorithm, the formula for converting FP32 to INT8 is as follows: ,in = (Global maximum value of single batch input tensor - Global minimum value) / 255, When overflow occurs, the computation is truncated to the extreme range of INT8 [-128, 127]. This is used when there are no available computing cores for the optimal and suboptimal adapted core types. The task is then dynamically quantized and downgraded before being scheduled to be executed on a non-adapted core. Specifically, when there are no valid pre-request requests for any adapted core type and there are non-adapted cores with a load rate lower than the preset overload threshold (80% for general cores and 85% for tensor / dedicated cores), cross-type flexible scheduling is triggered. At the same time, based on the precision sensitivity in the operator attribute label, a lightweight quantization module is started to dynamically convert the operator from a high-precision format (such as FP32) to a low-precision format (such as INT8) that is adapted to the computing power characteristics of the non-adapted cores. The downgraded atomic computing task is generated and assigned to the corresponding non-adapted core. Precision fallback rule: If the operator execution deviation is >5%, the execution is immediately terminated and the task is rolled back to an adapted core to wait for scheduling, ensuring system precision.

[0028] The precision compensation management unit receives in real time the non-optimal adapted kernel execution operator deviation data reported by the execution monitoring and deviation acquisition unit in the kernel-level scheduling agent. The data frame has a fixed format: operator ID (8 bits), kernel type ID (4 bits), input tensor feature value X (FP32), ideal calculated value (FP32), actual calculated value (FP32), and relative error (FP32, precision 0.001). The receiving module has a built-in CRC check to automatically remove abnormal data with format errors and out-of-range data to ensure the validity of the input samples. Then, the valid data is deduplicated and normalized to the [0,1] interval. Ten or more consecutive valid historical samples are selected as the fitting data source. When the sample size is insufficient, the historical mean compensation of the same type of operator is used as a fallback to avoid fitting failure and obtain preprocessed data. Then, the operator type + kernel type are used as the unique key. A one-to-one deviation mapping table is constructed for operator-kernel types to quickly index the compensation parameters of the target operator: specifically, a division hash function is used. The hash table has a physical capacity of ≥1024 entries, supports full-scenario operator coverage, and uses a chained hash table hardware implementation for hash collisions. The maximum length of the collision chain is ≤8. The entire query is completed by combinational logic circuits, and the latency of a single query is between 1 and 2 system clock cycles. Each mapping table entry contains an operator ID, a kernel type ID, a set of historical input values, a set of historical deviation values, a fitting coefficient A, a fitting coefficient B, a compensation activation flag, and a goodness-of-fit R². Based on the preprocessed bias samples, the least squares method is used to fit the bias mapping table, and the formula is: ,in Input values ​​for the operator, As compensation value, This is the weighting compensation coefficient. As the bias adjustment amount, the accuracy compensation management unit integrates a dedicated least-squares fitting hardware circuit. The circuit adopts a deep pipelined pulsating array structure, which can calculate sample statistics in parallel and quickly solve the coefficients through dedicated dividers and subtractors. The single fitting calculation delay is a fixed 15 system clock cycles; when the goodness of fit R² ≥ 0.95, the coefficients A and B are considered valid and written into the mapping table; The instruction is issued to the local task scheduling unit of the corresponding kernel-level scheduling agent, with a compensation accuracy ≥ 0.001, compatible with the kernel's execution timing. After receiving the instruction, the kernel-level scheduling agent will execute it according to the following when executing subsequent operators of the same type: The system corrects calculation results in real time without additional computational overhead. After the kernel-level scheduling agent completes the compensation execution, it sends the actual deviation after compensation back to the precision compensation management unit in real time, forming a closed-loop hardware process of deviation acquisition → data preprocessing → hash table indexing → least squares fitting → instruction issuance → compensation execution → deviation feedback. The system continuously iterates and updates the fitting coefficients A and B to ensure that the final calculation relative error is stable at ≤1%. The real-time load perception module includes a core-level load acquisition unit, an on-chip network congestion acquisition unit, a system-wide status view construction unit, and a threshold early warning unit, providing real-time data support for system-wide scheduling. The core-level load acquisition unit uses a moving average algorithm formula: average load = (current sample value + previous 4 sample values) / 5 to perform data noise reduction. With a preset acquisition period (12 system clock cycles), the system collects load data of all computing cores through the status acquisition units of each core-level scheduling agent, including computing power utilization, task queue length, memory access bandwidth utilization, and power consumption data. After collection, the data undergoes noise reduction processing using a moving average algorithm, with the window size set to 5 acquisition periods to ensure the periodicity and comprehensiveness of load data acquisition. The data fusion algorithm of the on-chip network congestion acquisition unit is as follows: Congestion vector value = (bandwidth utilization × 0.5 + forwarding delay × 0.3 + cache backlog × 0.2) normalized value. With the same acquisition period (12 system clock cycles), the bandwidth utilization, forwarding delay and cache backlog data volume of each link of the on-chip interconnection network are collected. The acquisition accuracy of cache backlog data volume is 16 bytes. The on-chip congestion vector of the entire link is generated through the data fusion algorithm. The vector dimension is consistent with the number of on-chip links, which accurately reflects the communication status of the on-chip network. The system-wide status view construction unit builds a real-time updated system-wide core-level load view and on-chip network status view based on the collected load data and on-chip congestion vectors. The view adopts a visual storage method, marking the idle (load rate ≤ 40%), normal (40% < load rate ≤ 80%), and overload (load rate > 80%) status of each computing core, as well as the idle (bandwidth utilization ≤ 50%) and congested (bandwidth utilization > 50%) status of on-chip links, providing intuitive system status basis for scheduling decisions; The threshold warning unit is used to generate warning signals to the cross-core collaborative arbitration module and the global task preprocessing module when the overload of the computing core or the congestion of the network link exceeds the threshold, so as to trigger task migration or task granularity adjustment. Specifically, when the load rate of the computing core exceeds the preset overload threshold (80%) or the congestion level of the on-chip link exceeds the preset threshold (50% bandwidth utilization), the corresponding overload warning signal and congestion warning signal are generated. The signals are represented by level signals (high level is warning) and synchronized to the cross-core collaborative arbitration module and the global task preprocessing module to trigger the migration scheduling of overloaded tasks or the dynamic adjustment of task fragmentation granularity, so as to ensure the stable operation of the system.

[0029] In this embodiment, key operating parameters (including weights and thresholds involved in modules such as the dynamic fragmentation adjustment unit, the computation cluster reorganization unit, and the topology-aware adaptation arbitration unit) were all pre-calibrated using an offline simulation platform. The calibration method is as follows: Simulation platform: SystemC is used to build a transaction-level chip architecture model, and VitisHLS is used to generate behavioral models of each computing core to build a complete heterogeneous multi-core system simulation environment. The on-chip network is simulated accurately using BookSim2.0. Benchmark load: The benchmark set used for calibration covers typical computation graphs of computer vision (such as ResNet50, YOLOv5), natural language processing (such as BERT-base, GPT-2 small model) and scientific computing (such as matrix factorization, FFT), ensuring coverage of scalar, vector, tensor and special computation modes; Optimization Objectives and Convergence Criteria: The calibration process uses the system's average computing power utilization and the end-to-end latency of the task as joint optimization objectives. Parameter search is performed using a genetic algorithm or Bayesian optimization. Iterative optimization continues until convergence occurs after 100 consecutive simulations where the standard deviation of the end-to-end latency fluctuation is ≤5% and the system's average computing power utilization is ≥90%. The resulting parameter set is the calibration result. Example of key parameter calibration: Communication cost weight ( Under various network load injection modes, the calibration was performed with the goal of minimizing the correlation coefficient between "communication overhead prediction error" and "actual scheduling delay", yielding α=0.6 and β=0.4. Dynamic fragmentation threshold: It is calibrated with "overall system throughput" as the optimization target. When the communication cost and the computing benefit are equal (ratio = 1), it is used as the decision point for splitting / merging. When the average bandwidth utilization of the on-chip network link exceeds 50%, it is determined that the network has entered a congested state, triggering task merging. Cluster reorganization threshold calculation: Statistical analysis revealed that when strong data dependency operator pairs (dependency strength > 0.8) are scheduled to different physical clusters, their communication latency accounts for an average of more than 40% of the total execution latency. Therefore, the threshold is set to 0.8; reorganization is triggered before such communication overhead becomes significant. In scenarios dominated by Transformer-type tasks, this threshold can be adjusted to 0.85. Topology-aware arbitration weights: These weights are calibrated with the objective of minimizing the average task completion time, resulting in a set of example weights, such as... In actual deployment, it can be fine-tuned based on specific load characteristics.

[0030] The above are merely embodiments of the present invention. The circuits, electronic components, and modules involved are all prior art, fully achievable by those skilled in the art, and require no further explanation. The scope of protection in this application does not involve improvements to the software and methods. Commonly known structures and characteristics in the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all prior art in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.

Claims

1. A high-efficiency collaborative artificial intelligence multi-core computing device, characterized in that, include: A heterogeneous multi-core computing cluster consists of multiple computing cores with different architectures deployed on different physical clusters on a chip; The on-chip interconnect network module connects the various modules and computing cores of the heterogeneous multi-core computing cluster for data interaction and instruction transmission; The global task preprocessing module is used to receive artificial intelligence computing tasks, parse the tasks based on the intermediate representation of the computation graph and the real-time execution status, and label the operators with attribute tags based on the computational characteristics, parallelism and memory access characteristics of the operators. Based on the real-time congestion status of the on-chip interconnect network, the splitting granularity of the operator is dynamically adjusted to generate atomic computing tasks with attribute tags, or multiple atomic computing tasks with data dependency strength exceeding a preset threshold are reorganized into computing cluster tasks with forced same-cluster scheduling flags. The distributed core-level scheduling agent cluster includes a core-level scheduling agent corresponding to each computing core. It is used to collect the real-time status of the corresponding computing core, predict the task completion time based on the current task execution progress, and send a pre-request request carrying the computing core status and the predicted completion time to the cross-core collaborative arbitration module before the task is completed. The cross-core collaborative arbitration module is used to perform dynamic arbitration, matching, and pre-distribution of tasks based on the attribute tags of the atomic computing tasks or computing cluster tasks and the pre-claim requests of each core-level scheduling agent. The real-time load awareness module is used to periodically collect load data from all computing cores and congestion data from each link of the on-chip interconnect network, and to build and update the overall system status view.

2. The high-efficiency collaborative artificial intelligence multi-core computing device according to claim 1, characterized in that, The global task preprocessing module includes: A dynamic fragmentation adjustment unit is used to dynamically adjust the operator splitting granularity based on the on-chip interconnection network congestion vector by evaluating communication costs and computational benefits. The communication cost is calculated based on the weighted sum of link delay and bandwidth utilization, and the computational benefits are quantified based on the parallel execution efficiency of the operators. When the ratio of communication cost to computational benefits is greater than a first threshold, operators are merged to increase granularity; when the ratio is less than or equal to the first threshold and network congestion is lower than a second threshold, operators are split to decrease granularity. The computation cluster reorganization unit is used to reorganize the multiple atomic computation tasks into a computation cluster task when the data dependency strength between multiple atomic computation tasks exceeds a third threshold, and to mark the computation cluster task with a forced same-cluster scheduling identifier.

3. The high-efficiency collaborative artificial intelligence multi-core computing device according to claim 2, characterized in that, Each of the aforementioned kernel-level scheduling agents includes: The shadow scheduling unit is used to predict task completion time based on the current task execution progress using a linear regression model; The task pre-claim unit is used to send the pre-claim request to the cross-core collaborative arbitration module within a preset time window before the predicted task completion time.

4. The high-efficiency collaborative artificial intelligence multi-core computing device according to claim 3, characterized in that, The cross-core collaborative arbitration module includes: The topology-aware adaptation arbitration unit is used to match the best computing core for the task. For computing cluster tasks, it prioritizes matching idle computing core groups within the same physical cluster. For atomic computing tasks, it selects the computing core with a load below the idle threshold and the closest physical distance as candidates based on the adaptation core type in its attribute label, and uses a weighted multidimensional scoring algorithm to determine the target scheduling core from the candidate cores. The task pre-distribution unit is used to pre-distribute the new tasks that have been arbitrated and matched, along with their corresponding data indexes, to the core-level scheduling agent of the target computing core before the current task of the target computing core is completed, so as to trigger the data prefetching operation.

5. The high-efficiency collaborative artificial intelligence multi-core computing device according to claim 4, characterized in that, The cross-core collaborative arbitration module also includes: The precision-driven flexible scheduling unit is used to schedule atomic computing tasks to non-adaptive cores for execution when there are no available computing cores for both the optimal and suboptimal adapted core types. The precision compensation management unit is used to collect the result deviation data of tasks executed on non-adapted cores, use the least squares method to fit and generate precision compensation parameters based on historical deviation data, and send precision compensation instructions to the core-level scheduling agent.

6. The high-efficiency collaborative artificial intelligence multi-core computing device according to claim 5, characterized in that, The real-time load sensing module includes: The threshold warning unit is used to generate a warning signal and send it to the cross-core collaborative arbitration module and the global task preprocessing module when the load rate of any computing core exceeds the overload threshold or the congestion level of any network link exceeds the congestion threshold, so as to trigger task migration or task splitting granularity adjustment.

7. A highly efficient collaborative artificial intelligence multi-core computing method, characterized in that, The method, applied to any one of the claims 1-6, comprises: S1. Dynamic task preprocessing: Parse the input computation task, label the operators with attribute tags, and dynamically adjust the operator splitting granularity based on the real-time network status to generate atomic computation tasks, or reorganize strongly data-dependent atomic computation tasks into computation cluster tasks. S2. Distributed pre-request: The scheduling agent of each computing core monitors the status of its own core in real time and predicts the completion time of the current task. Before the task is completed, it sends a pre-request request to the cross-core collaborative arbitration module. S3. Topology-aware arbitration and pre-distribution: The cross-core collaborative arbitration module matches the best computing core for a task based on task attributes, pre-request requests, overall system load, and physical topology, and pre-distributes the new task and data index before the current task on that core ends. S4. Flexible scheduling and accuracy compensation: When no suitable core is available, the task is dynamically quantified and downgraded to be scheduled to be executed on a non-suitable core, and the execution deviation is collected. Based on the historical deviation, the compensation parameters are fitted to perform closed-loop accuracy compensation for subsequent similar tasks. S5. Task Execution and Status Update: The computing core executes the assigned tasks, reports the status upon completion, triggers a new round of pre-claiming, and updates the load view of the entire system.

8. The efficient collaborative artificial intelligence multi-core computing method according to claim 7, characterized in that, In step S1, the dynamic adjustment of operator splitting granularity includes: evaluating communication costs based on on-chip network congestion vectors, evaluating computational benefits based on operator parallel potential, and deciding whether to merge or split operators based on the ratio of communication costs to computational benefits.

9. The efficient collaborative artificial intelligence multi-core computing method according to claim 7, characterized in that, In step S3, matching the best computing core for the atomic computing task includes: firstly, screening computing cores with a load below the idle threshold among the optimal matching core types; if there are multiple, selecting the computing core with the lightest load and the closest physical distance to the task data dependent core.

10. The efficient collaborative artificial intelligence multi-core computing method according to claim 7, characterized in that, In step S4, the closed-loop accuracy compensation includes: establishing a deviation mapping table with operator type and execution kernel type as keys, using the least squares method to fit the deviation pattern of the operator executed on a specific kernel, obtaining the weight compensation coefficient and the bias adjustment amount, and generating a compensation instruction accordingly, and performing real-time correction in subsequent execution according to "compensated result = original result × weight compensation coefficient + bias adjustment amount".

Images