An AI server data processing optimization system and method based on distributed heterogeneous computing

Abstract

The application discloses an AI server data processing optimization system and method based on distributed heterogeneous computing, and relates to the technical field of artificial intelligence and distributed heterogeneous computing, and comprises a distributed heterogeneous computing cluster, a dynamic cooperative scheduling subsystem and a double-layer closed-loop verification subsystem; characterized in that the distributed heterogeneous computing cluster comprises a plurality of interconnected computing nodes, and each node is integrated with at least two hardware units. The AI server data processing optimization system and method based on distributed heterogeneous computing realize deep cooperative optimization of computing resource scheduling and data flow transmission by constructing a hardware-data-task tripartite graph model and a time-space cooperative propagation algorithm, so as to reduce end-to-end processing delay and maximize the utilization rate of heterogeneous resources.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and distributed heterogeneous computing, and specifically to an AI server data processing optimization system and method based on distributed heterogeneous computing. Background Art

[0002] With the explosive growth of AI applications in cloud-based inference, training, and edge computing, AI server clusters are required to handle massive, dynamic, and compute-intensive data loads. To improve processing efficiency, modern AI servers widely adopt distributed heterogeneous computing architectures. This architecture integrates multiple computing units, including CPUs, GPUs, FPGAs, and various specialized AI accelerator chips, and connects multiple server nodes via high-speed networks for coordinated operation. This architecture aims to leverage the specialized capabilities of different hardware units and, in theory, significantly improves system throughput and energy efficiency. However, existing technical solutions face a fundamental challenge in achieving efficient data processing optimization: in a distributed, heterogeneous environment, the dynamic scheduling of computing tasks lacks deep coordination between data flows across devices and nodes, leading to severe resource imbalances and end-to-end processing delays. Specifically, existing scheduling strategies often focus on local load balancing of computing resources while ignoring the global nature and real-time requirements of data flows. AI data processing tasks often involve complex forward and backward dependencies, forming data flow graphs, and task characteristics dynamically change with input data. Existing methods fail to fully understand these dynamic task characteristics and the resulting data dependencies, resulting in rigid scheduling decisions. The consequences are: powerful heterogeneous units may sit idle, awaiting data generated by their dependent predecessor tasks; tasks may be assigned to nodes far from the data they need, causing high-latency data migration; or communication-intensive subtasks may be incorrectly scheduled to locations with limited network bandwidth. This disconnect between computing resources and data flow prevents the efficient and coordinated utilization of valuable heterogeneous computing power. Overall system performance is limited by communication bottlenecks and idle resources, severely restricting the potential of distributed heterogeneous architectures and failing to meet the growing demand for low-latency, high-throughput AI data processing. Summary of the Invention

[0003] In response to the shortcomings of the existing technology, the present invention provides an AI server data processing optimization system and method based on distributed heterogeneous computing, which solves the problem of how to achieve deep collaborative optimization of dynamic scheduling of computing resources and real-time transmission of data streams to reduce end-to-end processing delays and maximize heterogeneous resource utilization.

[0004] To achieve the above objectives, the present invention is implemented through the following technical solutions: an AI server data processing optimization system based on distributed heterogeneous computing, comprising: a distributed heterogeneous computing cluster, a dynamic collaborative scheduling subsystem, and a two-layer closed-loop verification subsystem;

[0005] The distributed heterogeneous computing cluster includes a plurality of interconnected computing nodes, each node integrating at least two hardware units;

[0006] The dynamic collaborative scheduling subsystem is connected to the heterogeneous resource monitoring module and the task dependency parsing module to construct a hardware-data-task ternary graph model and perform collaborative scheduling;

[0007] The double-layer closed-loop verification subsystem is connected to the dynamic collaborative scheduling subsystem and includes a real-time verification layer and a feedback verification layer.

[0008] It should be further explained that during the specific implementation process, when the system starts, the computing nodes in the distributed heterogeneous computing cluster are interconnected through a high-speed network, and the built-in heterogeneous resource monitoring module of each node continuously collects the real-time computing power utilization, memory bandwidth occupancy and cross-device communication delay data of the CPU, GPU, FPGA and AI acceleration chip; at the same time, the task dependency parsing module receives the AI ​​data stream to be processed, dynamically parses the dependency relationship between tasks and generates a directed acyclic graph. The nodes in the graph represent atomic tasks, and the directed edges represent the task execution order and data transmission path.

[0009] Based on these inputs, the dynamic collaborative scheduling subsystem constructs a hardware-data-task ternary graph model: physical compute nodes are abstracted as hardware nodes, whose vectors dynamically integrate current computing power and communication latency metrics; the physical locations where data is stored are abstracted as data nodes, whose vectors encode data size, access popularity, and node location; and AI data processing tasks are abstracted as task nodes, whose vectors contain computational model characteristics and the depth of their dependencies within the task's directed acyclic graph. These three types of nodes are connected by three types of directed edges: edge weights between hardware nodes reflect network transmission latency, edge weights between task nodes reflect data dependency strength, and edge weights between task nodes and hardware nodes reflect the expected execution efficiency of the task on that hardware.

[0010] The timing cooperative propagation algorithm is run during the scheduling process:

[0011] Forward propagation phase: Starting from the root task node with no dependencies, the earliest start time of each subtask is predicted layer by layer along the task dependency edges, and an initial scheduling plan is generated based on the execution efficiency of the bound hardware nodes.

[0012] Backward propagation: Tracing back from the final task node, if a hardware node is detected to be idle due to unready data, the system traces back to the data source node along the hardware communication edge, dynamically shortening the data prefetch window or switching the prefetch path. For example, data originally transmitted via Ethernet can be directly connected to the accelerator memory via NVLink. The data prefetch path switching condition is: if back propagation detects that data prefetch delays will cause the hardware idle time to exceed 50% of the task's computation time, the backup high-speed link is automatically activated.

[0013] When the hardware resource monitoring module detects that the actual delay of a critical path task exceeds a threshold, it triggers tearing rescheduling, which includes deconstructing only the subgraph associated with the task in the ternary graph model, including its direct predecessor / successor task nodes and bound hardware nodes, and re-executing the spatiotemporal collaborative propagation algorithm within the subgraph to generate a local optimization solution. The scheduling of non-critical path tasks remains unchanged. The tearing rescheduling scope is determined as follows: subgraph deconstruction is activated only when the delayed task is on the critical path of the current task graph and the delay affects the global progress by more than 3%.

[0014] The two-layer closed-loop verification subsystem operates synchronously: the real-time verification layer replicates the production environment topology in an independent sandbox, injects shadow data streams, and executes candidate scheduling solutions in parallel. If the actual hardware idle rate deviates from the predicted value by more than 5% or the data arrival latency deviates by more than 10 milliseconds, the solution is discarded.

[0015] The feedback verification layer captures the optimal execution trajectory of the production environment and injects two types of perturbations through the negative sample adversarial training engine: simulating a sudden drop in network bandwidth of more than 30% or a random hardware unit frequency reduction of 20%. The post-perturbation performance decay data is used to reversely train the vector generation module of the ternary graph model, improving its tolerance to abnormal operating conditions. The trained parameters are fed back to the scheduling subsystem in real time. During this period, the intensity of the negative sample perturbation is controlled as follows: network congestion simulation uses an exponential decay model, and the duration of the hardware frequency reduction perturbation does not exceed 20% of the task execution period. The vector generation logic is as follows: hardware node vectors are immediately reconstructed when computing power fluctuations exceeding 15% are detected, and data node vectors are updated on a minute-by-minute basis as access popularity changes.

[0016] Preferably, the heterogeneous resource monitoring module collects the computing power utilization, memory bandwidth occupancy and cross-device communication delay of the hardware unit in real time; the task dependency parsing module dynamically generates a directed acyclic graph of AI data processing tasks and identifies data dependencies.

[0017] Preferably, the hardware-data-task ternary graph model is constructed in the following manner:

[0018] Abstract computing nodes, storage locations, and data processing tasks into three types of graph nodes;

[0019] Dynamically generate vectors containing real-time computing power and communication delay for hardware nodes;

[0020] Generate a vector containing storage location and access popularity for the data node;

[0021] Generate a vector containing computation patterns and dependencies for task nodes;

[0022] Map hardware communication links, task data dependencies, and task hardware binding relationships into weighted directed edges.

[0023] Preferably, the coordinated scheduling includes a spatiotemporal coordinated propagation algorithm:

[0024] Forward propagation phase: predict the earliest start time of subtasks along the task dependency edges and generate an initial scheduling plan;

[0025] Backpropagation phase: Feedback resource competition status along the hardware communication edge and dynamically adjust the data prefetch path.

[0026] Preferably, during the construction of the hardware-data-task ternary graph model, a tearing rescheduling mechanism is used to handle task delays, including: when the delay of the critical path task exceeds a threshold, only deconstructing the associated subgraph; re-executing the spatiotemporal collaborative propagation algorithm within the subgraph range, and outputting a local optimization solution.

[0027] Preferably, the real-time verification layer performs the following operations:

[0028] Inject shadow data streams into the sandbox environment to test candidate solutions in parallel;

[0029] Compare the difference between actual and predicted hardware idle rates;

[0030] Compare the actual and predicted data arrival delay differences.

[0031] Preferably, the feedback validation layer includes a negative sample adversarial training engine:

[0032] Inject simulated network congestion and hardware frequency reduction disturbances based on historical optimal solutions;

[0033] Use the perturbed data to train the ternary graph model and update the model parameters to the real-time validation layer.

[0034] A method for optimizing AI server data processing based on distributed heterogeneous computing, comprising the following steps:

[0035] S1: Constructing a hardware-data-task ternary graph model;

[0036] S2: Generate a joint scheduling and data prefetching scheme through a spatiotemporal cooperative propagation algorithm;

[0037] S3: Perform two-layer closed-loop verification: real-time sandbox testing and negative sample adversarial training.

[0038] Preferably, the combined solution makes the error between the computing unit wake-up time and the data arrival time less than a predetermined time threshold; and the cross-node data migration amount is reduced by more than a predetermined ratio threshold compared with the non-optimized solution.

[0039] Preferably, the parameter update output by the negative sample adversarial training directly drives the vector generation strategy iteration of the ternary graph model.

[0040] The present invention provides an AI server data processing optimization system and method based on distributed heterogeneous computing. It has the following beneficial effects:

[0041] (1) This AI server data processing optimization system and method based on distributed heterogeneous computing achieves deep collaborative optimization of computing resource scheduling and data flow transmission by constructing a hardware-data-task ternary graph model and a spatiotemporal collaborative propagation algorithm. Through the dynamic collaborative scheduling subsystem based on real-time resource monitoring and task dependency analysis, computing nodes, data storage, and processing tasks are abstracted into graph nodes. Hardware communication, task dependencies, and binding relationships are mapped through weighted directed edges. Forward propagation is used to generate the initial scheduling plan, and backpropagation is used to dynamically adjust the data prefetch path. The tearing rescheduling mechanism is combined to locally optimize the critical path delay. This mechanism enables the error between the wake-up time of the computing unit and the data arrival time to be controlled, the amount of data migration across nodes is reduced, the utilization rate of heterogeneous resources is improved, and hardware idleness caused by data dependency waiting or cross-node transmission delay is avoided.

[0042] (2) This AI server data processing optimization system and method based on distributed heterogeneous computing forms a self-reinforcing feedback mechanism through real-time sandbox testing and negative sample adversarial training. The real-time verification layer injects shadow data streams in an isolated environment to parallel test candidate solutions, ensuring the feasibility of the scheduling solution by comparing hardware idle rates with data delay predictions. The feedback verification layer injects disturbances such as simulated network congestion and hardware frequency reduction based on the historical optimal solution, uses the generated negative sample data to train the ternary graph model, and dynamically updates parameters such as hardware vectors and communication edge weights to improve the system's robustness to abnormal working conditions. This "test-train-iterate" closed-loop mechanism enables the system to adapt to dynamic load changes and sudden failures in distributed heterogeneous environments, continuously optimize scheduling strategies, and form a complete intelligent optimization chain from data perception, collaborative scheduling, to verification feedback, providing an excellent solution for high-throughput, low-latency AI data processing needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the overall framework of the present invention;

[0044] Figure 2 This is a control logic timing diagram of the present invention. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0046] See also Figure 1 and Figure 2 , the present invention provides a technical solution: an AI server data processing optimization system based on distributed heterogeneous computing, comprising: a distributed heterogeneous computing cluster, a dynamic collaborative scheduling subsystem and a two-layer closed-loop verification subsystem;

[0047] A distributed heterogeneous computing cluster consists of multiple interconnected computing nodes, each of which integrates at least two hardware units;

[0048] The dynamic collaborative scheduling subsystem connects the heterogeneous resource monitoring module and the task dependency analysis module to build a hardware-data-task ternary graph model and perform collaborative scheduling;

[0049] The two-layer closed-loop verification subsystem is connected to the dynamic collaborative scheduling subsystem, including a real-time verification layer and a feedback verification layer.

[0050] The heterogeneous resource monitoring module collects the computing power utilization, memory bandwidth occupancy and cross-device communication delay of hardware units in real time; the task dependency analysis module dynamically generates a directed acyclic graph of AI data processing tasks and identifies data dependencies.

[0051] It should be further explained that, during the specific implementation process, when the heterogeneous resource monitoring module is in continuous operation, differentiated monitoring strategies are implemented for the different hardware units integrated in each computing node: for the GPU unit, its SM occupancy rate and memory bandwidth usage ratio are captured in real time. When it is detected that the context switching frequency caused by multi-task queuing exceeds 50 times per second, the sampling frequency is automatically increased to the millisecond level; for the FPGA unit, the logic unit utilization rate and DDR access delay are monitored in particular. If the queue depth of the memory control unit is found to continue to increase, the hardware is marked as oversaturated. Among them, the hardware oversaturation response condition is: when the GPU memory bandwidth occupancy rate is greater than 95% for 200ms, the monitoring module marks the "resource overload" state, and the task resolution module automatically downgrades the new task binding weight of the node by 50%.

[0052] Cross-device communication latency monitoring covers three types of channels: PCIe, NVLink, and RDMA. When inter-node transmission latency increases suddenly and the packet retransmission rate exceeds 5%, the network link is identified as abnormal and the topological path is recorded. All monitoring data is aggregated to the dynamic collaborative scheduling subsystem in a timestamp-aligned manner. Network anomaly isolation includes: When an RDMA path retransmission rate is detected to be greater than 8% for three consecutive samples, the monitoring module amplifies the communication latency value by three times and reports it, triggering the analysis module to avoid task binding for that path.

[0053] When the task dependency parsing module processes the input AI data stream, it first builds the initial dependency relationship based on the input and output declarations in the task metadata, and dynamically expands the dependency graph in the streaming computing scenario: when the newly arrived task has data overlap with the executing task, the data dependency edge is immediately inserted; when the task output is referenced by multiple downstream tasks, a virtual aggregation node is automatically created to avoid duplicate data transmission. For task flows with conditional branches, the module maintains probabilistic dependency edges and prunes invalid paths in real time after the branch is determined. When a hardware failure or priority change event is triggered, the module reconstructs the affected task subgraph: if a GPU node fails, its bound task edges are unbound and the execution weight of each hardware node is re-evaluated; if a high-priority task is inserted, its predecessor task is forced to adopt preemptive scheduling and the dependency path depth is compressed.

[0054] The two modules operate in tandem: When the resource monitoring module detects a hardware status change, it immediately pushes an event to the task analysis module. The task analysis module then removes the edge weight constraints on the hardware-bound task, triggering the scheduling subsystem to reassign tasks. Conversely, when the analysis module identifies a data-intensive task chain, it issues instructions to the monitoring module to enable cache hit rate monitoring for associated storage nodes and increase the network bandwidth sampling frequency. In distributed training scenarios, gradient aggregation tasks are identified as critical path nodes. The monitoring module implements microsecond-level latency tracking on the bound AllReduce communication link, while the analysis module locks the task's forward propagation dependency chain to prevent rescheduling. The dynamic dependency handling process is as follows: When a new node is added to the streaming task graph, if its input port matches the output port of a running task at least 80%, a strong dependency edge is established; otherwise, a weak dependency edge is created and data validation is initiated. For tasks marked with a priority level ≥8, such as real-time inference requests, the analysis module freezes the subgraph structure of its immediate predecessor task, and the monitoring module implements read-only monitoring of the hardware resources associated with this subgraph.

[0055] The hardware-data-task ternary graph model is constructed in the following way:

[0056] Abstract computing nodes, storage locations, and data processing tasks into three types of graph nodes;

[0057] Dynamically generate vectors containing real-time computing power and communication delay for hardware nodes;

[0058] Generate a vector containing storage location and access popularity for the data node;

[0059] Generate a vector containing computation patterns and dependencies for task nodes;

[0060] Map hardware communication links, task data dependencies, and task hardware binding relationships into weighted directed edges.

[0061] It should be further explained that in the specific implementation process, when constructing the hardware-data-task ternary graph model, the system first performs an abstract mapping of the physical entities, including the following:

[0062] Hardware node generation rules: Each physical computing unit is mapped to an independent node, and its real-time vector is dynamically synthesized according to the following logic: When the monitoring module reports that the GPU memory bandwidth utilization rate is greater than 90%, the value of the "Computing Availability" dimension in the vector is reduced by 30%; if the FPGA's DDR access latency suddenly increases by more than 50%, the "Memory Efficiency" dimension is marked as abnormal; the status of the cross-node communication link directly affects the "Network Affinity" dimension. For example, if the RDMA path retransmission rate is detected to be greater than 5%, the value of this dimension is reset to zero.

[0063] Data node creation mechanism: Each data shard in distributed storage corresponds to a data node, and its vector is dynamically updated based on the access pattern: when it is requested by more than three tasks within 10 seconds, the "hotness" dimension is linearly increased to the maximum value; if the data shard is stored across nodes, the "location" dimension records the topological path cost, and the location identifier is automatically switched when the access latency of the backup copy exceeds 200% of the primary copy. The trigger conditions for the dynamic update of the vector include the following:

[0064] Hardware node vector: Immediate reconstruction when computing power fluctuation > 15% or communication delay change > 20%;

[0065] Data node vector: updated when access frequency change rate > 50% / minute or location changes;

[0066] Task node vector: recalculated when task execution fails or priority is adjusted.

[0067] The task node binding logic includes: extracting key attributes from the task directed acyclic graph (DAG) of atomic task node vectors; marking tasks with a recursion depth greater than 5 layers with a "high dependency" feature; and activating the "distributed" flag for tasks spanning more than 4 hardware units. The binding edge weights between task nodes and hardware nodes are initialized based on historical execution data: if a convolution task executes three times faster on a GPU than on an FPGA, the GPU binding edge weight is set to three times that of the FPGA.

[0068] The three types of dynamic maintenance strategies for directed edges include the following:

[0069] 1. Hardware communication edge: Initial weight = baseline latency × real-time congestion factor. When the monitored path packet loss rate exceeds 2% for three consecutive samples, the congestion factor is increased from 1.0 to 2.5. If the NVLink direct connection channel is idle, a preferred edge with a weight of 0.3 times the baseline value is automatically created. Optimized communication edge path switching includes: When the edge weight exceeds the preset threshold and an NVLink path exists, a new edge is automatically created and the original edge is discarded. For example, if the Ethernet path weight is greater than 1.2 and an NVLink path weight is ≤0.5, a new edge is automatically created and the original edge is discarded. If the new path fails to transmit twice in a row, the original path is restored with a 50% penalty weight.

[0070] 2. Task dependency edge: Weight = data volume / transmission rate. When the compression rate of the upstream task output data is greater than 60%, the weight value decreases according to the compression ratio. If the downstream task triggers retransmission due to input data verification failure, the dependency edge weight is temporarily doubled.

[0071] 3. Task hardware binding edge: Weight = 1 / expected execution time. When the actual execution time deviates from the predicted value by more than 15% twice in a row, the weight reconstruction is triggered: if the actual execution time is faster, the weight increases by the deviation percentage; otherwise, the weight decreases.

[0072] Abnormal working condition response mechanism includes the following:

[0073] Hardware node failure: When a GPU node's temperature exceeds the threshold and is forced to reduce its frequency, all its bound edge weights are reset to zero, and the associated tasks are automatically migrated to suboptimal hardware. The edge weight correction rules include: when a task is first executed on a hardware unit, the weight is equal to the inverse of the average duration of similar tasks. After three executions, a weighted moving average is used, that is, the new weight = 0.7 × historical average + 0.3 × the inverse of the current duration.

[0074] Data location mutation: When a storage node disk failure causes data replica migration, the associated data nodes are automatically scanned to update the location vector, while the original dependency edges are released and a new path is rebuilt.

[0075] Task dependency breaking: In streaming computing, if the upstream task times out and the system detects that the downstream task input port times out and does not receive data, the dependency edge is released and the data reconstruction subtask is started.

[0076] Co-scheduling includes spatio-temporal cooperative communication algorithms:

[0077] Forward propagation phase: predict the earliest start time of subtasks along the task dependency edges and generate an initial scheduling plan;

[0078] Backward propagation: Feedback on resource contention along hardware communication edges dynamically adjusts data prefetching paths. It should be noted that, in its implementation, when executing the spatiotemporal co-propagation algorithm, the system initializes the scheduling framework with forward propagation: Starting from the root node of the task directed acyclic graph, the system calculates data transfer times based on the task dependency edge weights and uses the target hardware node's execution efficiency vector to predict the earliest subtask start time. For example, if an image preprocessing task requires transferring 10GB of data to a GPU node, and the dependency edge weight indicates a transmission rate of 5GB / s and the GPU's current execution efficiency is 0.8, the predicted transmission time is 2 seconds and the execution time is 1.25 seconds. The earliest start time for this task is the end time of the upstream task + 2 seconds. The initial scheduling plan is generated layer by layer according to this rule, ensuring that tasks without dependency conflicts are executed in parallel to the greatest extent possible. The priority for parameter conflict arbitration is: end-to-end latency optimization > migration reduction > resource utilization improvement.

[0079] During the backpropagation phase, data flow defects are corrected in real time: scanning backward from the last-level task node, if it is detected that the idle rate of the task ready queue of the GPU node is greater than 30%, it is determined that there is a data supply delay. At this time, the hardware communication edge is traced back to the storage node where the data is located: if the data node is located in the NVLink topology of the same cabinet and the communication edge weight is less than 0.4, the high-speed channel is forced to enable pre-fetching of data; if the data needs to be transmitted across cabinets and the communication edge weight is greater than 1.2, data sharding compression or switching to the nearest replica is initiated. When the cumulative idle time of the hardware node due to data waiting exceeds 40% of the task calculation time, the current binding edge is automatically released and the suboptimal hardware is re-evaluated. The conditions for releasing the binding edge are: the hardware node has two consecutive task execution timeouts greater than 30% of the predicted value; the hardware utilization rate is less than 50% for 5 task cycles due to data waiting.

[0080] Dynamic response to network fluctuation strategy: Continuously monitor hardware communication edge status changes during the backpropagation process. When the RDMA path packet loss rate suddenly increases to more than 8%, the data prefetch operation associated with the path is immediately frozen, and the alternative path weight evaluation is started: if the InfiniBand backup path communication edge weight is less than 60% of the original path, the prefetch path is switched and the dependent edge weight is reset; if the weights of all backup paths are degraded, the data reconstruction task is triggered to generate a copy locally on the target node. For time-sensitive tasks, such as real-time inference requests, when backpropagation detects the risk of data prefetch delay, it directly seizes the allocated network bandwidth of low-priority tasks to ensure the completion of critical data stream transmission. Among them, data prefetch optimization cascade response, including the first response, the second response and the third response. The first response is to enable the same-node memory copy, the second response is to switch to the copy in the same NUMA domain, and the third response is to start data compression during cross-domain transmission.

[0081] Abnormal working condition response mechanism includes the following:

[0082] Data readiness misjudgment handling: When the forward propagation predicted data arrival time deviates by more than 15% from the actual time, the backpropagation module automatically traces the source of the deviation. If the cause is an inaccurate dependent edge weight, such as a 30% drop in the actual transmission rate compared to the predicted value, a 150% weight penalty is applied to that edge. If the cause is hardware node execution delay, such as a sudden GPU thermal throttling, the priority of the bound edge weight at that node is reduced. The forward propagation time prediction rules include: the earliest start time of the root task is equal to the current system timestamp; the earliest start time of the downstream task is equal to MAX(the end time of all upstream tasks) + the transmission time of the dependent edge; and the task end time is equal to the earliest start time + (task computational load / hardware node execution efficiency).

[0083] Hardware resource contention arbitration: When multiple reverse propagation instructions simultaneously apply for high-speed channels, such as NVLink, the system arbitrates based on task priority and latency impact: if the priority is the same, the task with the smaller data volume will be given priority. The impact is calculated as the task's dependency depth multiplied by the latency duration. The deeper the depth and the longer the latency, the higher the priority.

[0084] Cross-node data conflict resolution: If two backpropagation instructions require the migration of the same data shard to different nodes, such as Task A to Node 1 and Task B to Node 3, the system checks the strength of data dependencies: Data replicas that are relied upon by more downstream tasks receive migration priority, while remaining replicas are converted to remote access and have their communication edges marked as weighted. Backpropagation trigger conditions include: the hardware node queue idle rate is >25% for three consecutive seconds; and the actual task start time is >200 milliseconds later than the predicted value.

[0085] Use tearing rescheduling mechanism to handle task delays:

[0086] When the critical path task delay exceeds the threshold, only the associated subgraph is deconstructed;

[0087] Re-execute the spatiotemporal co-propagation algorithm within the subgraph and output a local optimization solution.

[0088] It's important to note that, in practice, when the hardware resource monitoring module detects that the actual latency of a critical path task exceeds a preset threshold, the system triggers a tear-down rescheduling mechanism. This involves first locating the task's node in the ternary graph model, tracing back through all its direct predecessor task nodes to the nearest parallel branch point, and then scanning down through its direct successor task nodes to the convergence point, forming a closed subgraph. This subgraph only includes hardware nodes bound to these tasks; unrelated hardware is not included in the decomposition.

[0089] The spatiotemporal co-propagation algorithm is re-run within the subgraph. During the forward propagation phase, only the task dependencies within the subgraph are evaluated. If the predecessor task's output data is found to be compressible with a compression ratio greater than 40%, the dependency weights are updated and the transmission window is shortened. During the backward propagation phase, the focus is on optimizing the hardware communication paths within the subgraph. If the PCIe path weight from FPGA1 to GPU3 is greater than 1.5, an alternative path via NVLink bridging is automatically created and the data flow is migrated. Once a local optimization solution is generated, only the scheduling instructions for the corresponding portion of the atomic graph are replaced, while the scheduling parameters of non-critical tasks executing outside the subgraph remain unchanged.

[0090] The subgraph range determination rules are as follows:

[0091] Critical path tasks: tasks whose delay affects the overall progress by ≥3%;

[0092] Subgraph boundary: trace back to the first bifurcation point and extend downward to the first convergence point;

[0093] Hardware inclusion principle: only bind physical units of tasks within a subgraph.

[0094] Resource conflict handling during rescheduling: If subgraph optimization requires the use of global shared resources, and the resources have been locked by tasks outside the subgraph, the system starts priority arbitration: compare the dependency depth of the delayed task with the depth of the occupied task, and the priority increases by 2 times for each level of depth increase; if the depth is the same, the task with the shorter remaining execution time is selected to obtain resources first. When a hardware node in a subgraph needs to execute a new scheduling plan but is running a non-critical task, the system calculates the task interruption cost, which is equal to the execution time divided by the total estimated time. If the cost is less than 25%, it is forced to preempt, otherwise it waits until the current task checkpoint before intervening. The resource preemption threshold is calculated as follows:

[0095] Interruption cost = (computing power resources used by the task / total required computing power) × 100%;

[0096] Preemption is allowed if the cost is ≤ 25% or the task priority is ≤ 5 (1-10, 10 is the highest).

[0097] Abnormal working condition response mechanism includes the following:

[0098] Subgraph optimization failure handling: If local rescheduling fails to reduce latency to within a threshold, the system unblocks the subgraph, expands it to the next level to include more parallel branches, and re-executes the optimization process. If failure persists after expansion, the task quality requirements are downgraded to reduce computation time. The conditions for expansion failure include: latency reduction after the initial optimization is less than 30%; and the critical path impact remains greater than 2.5%.

[0099] Hardware resources are unavailable: If the optimization plan specifies migration to the FPGA2 node but the node suddenly fails, historical execution records are automatically enabled: the second most efficient available hardware for the task in the past 72 hours is selected, and interruptible low-priority tasks on the hardware are frozen.

[0100] Data consistency conflict: When rescheduling requires modifying the location of the distributed data replica, but the Node1 replica is being read by other tasks, the system starts the dual-replica transition mode: maintaining the original replica for external tasks, while creating a new replica to serve the tasks in the subgraph. After the external tasks are completed, the old replica is synchronously deleted. The path switching priority strategy includes:

[0101] First priority: same-node memory copy;

[0102] Second priority: shared memory path within the NUMA domain;

[0103] Third priority: cross-node RDMA direct connection.

[0104] The real-time verification layer performs the following operations: injects shadow data streams into the sandbox environment to test candidate solutions in parallel; compares the actual and predicted hardware idle rate differences; and compares the actual and predicted data arrival delay differences. It should be further explained that, during the specific implementation process, when the real-time verification layer operates, the hardware topology and software configuration of the production cluster are completely replicated in an independent sandbox environment, including heterogeneous computing unit types, network link bandwidth, and storage node distribution. The injected shadow data stream uses a desensitized copy of real business data, and its task dependencies evolve synchronously with the production environment. During the parallel execution of the candidate scheduling solutions in the sandbox, the verification engine simultaneously starts three sets of monitoring, including hardware idle rate tracking, data arrival delay monitoring, and cross-node transmission verification;

[0105] Hardware idle rate tracking: records the idle time ratio of the task queue of each GPU / FPGA unit. When the actual value deviates from the predicted value by more than 5% and lasts for three task cycles, it is marked as a "resource prediction failure" event; data arrival delay monitoring: captures the actual data ready timestamp at the task input port. If the delay is more than 10 milliseconds compared with the preset time of the scheduling plan, a "data flow break" alarm is triggered; cross-node transmission verification: compares the actual throughput of the RDMA link with the bandwidth quota applied by the plan. When the actual value is less than 70% of the quota and the data sharding verification failure rate is more than 3%, it is judged as "insufficient path reliability".

[0106] These three sets of monitoring results are dynamically fed into the decision-making engine. If any solution triggers "resource prediction failure" ≥2 times or "data flow disruption" ≥3 times within a single test cycle, it is immediately discarded. If all candidate solutions experience "insufficient path reliability" in two consecutive rounds of testing, network bandwidth constraints are automatically relaxed and a new solution is generated. The candidate elimination rules include: ≥2 resource prediction failures or ≥3 data flow disruptions within a single cycle → immediate discard; ≥2 consecutive cycles of insufficient path reliability → triggering solution regeneration; ≥50% delay increase during interference rejection testing → downgrade to a backup solution.

[0107] Abnormal operating condition response mechanisms include sandbox resource contention handling, shadow data flow distortion compensation, and network jitter simulation. Sandbox resource contention handling includes: when multiple candidate solutions require concurrent testing but physical resources are insufficient, the system sorts them by their generated timestamps, prioritizing verification of the most recent solution. Simultaneously, a lightweight simulation mode is activated, replacing real execution with an efficiency model for non-critical hardware nodes, leaving only the GPU / FPGA units on the critical path for full physical testing. Lightweight simulation trigger conditions include: the number of candidate solutions > the number of available GPU units × 2, and the simulation error rate for non-critical nodes < 8%. Criteria for determining an excellent anti-interference solution include: a latency increase of ≤20% at a 5% packet loss rate; and a task failure rate of ≤1 / 1000 at a 10% packet loss rate.

[0108] Shadow data loss distortion compensation includes: If encryption of desensitized data causes computational feature shifts, the monitoring module dynamically calibrates the predicted values. If the actual execution time of a task deviates by more than 15% from the historical benchmark, a proportional correction is applied to the predicted values ​​of all candidate solutions in the same batch. The data distortion compensation logic includes the following: calibration factor = (measured task duration / benchmark duration) × solution predicted value; this is only enabled when the deviation direction of solutions in the same batch is consistent (e.g., all measured values ​​> the benchmark value).

[0109] Network jitter simulation involves actively injecting controlled packet loss into the sandbox InfiniBand switch to test the stability of candidate solutions under this disturbance. Solutions with a latency increase of less than 20% compared to a undisturbed environment at a 5% packet loss rate are marked as "Excellent for Disturbance Resilience" and prioritized for output.

[0110] The feedback validation layer includes a negative adversarial training engine: based on the historical optimal solution, it injects simulated network congestion and hardware frequency reduction perturbations; uses the perturbed data to train a ternary graph model, and updates the model parameters to the real-time validation layer. It should be noted that during implementation, when the negative adversarial training engine is activated, the complete execution trajectory of the historical optimal scheduling solution is captured from the production environment, including hardware resource utilization curves, data flow transmission sequences, and task dependency graphs. The core operation of the engine is divided into three stages: disturbance factor injection, degradation scenario generation, and model adversarial training. Among them, disturbance factor injection includes network congestion simulation and hardware frequency reduction attack. Network congestion simulation includes: dynamically injecting controllable packet loss on the data stream transmission path. When the original solution uses the RDMA path, 30% of the data packets are randomly selected to impose a packet loss rate of 1% to 15%, and the weight of the associated communication edge is amplified by 2-5 times; hardware frequency reduction attack includes: simulating performance degradation of the GPU / FPGA unit bound to the solution. If the historical execution time average is T seconds, the hardware is forced to extend the execution time of the next cycle to 1.2T to 2.0T times, and block 20% to 50% of the memory bandwidth.

[0111] Degraded scenario generation involves executing the perturbation scenario and collecting performance degradation data. When a critical task latency increase of >40% or a heterogeneous resource utilization decrease of >35% is detected, the scenario is marked as a valid negative sample. For high-frequency failure modes, the intensity of similar perturbations is automatically increased, meaning that the subsequent frequency reduction ratio for the same hardware model is increased to 1.5 times the baseline value.

[0112] Among them, the judgment of effective negative samples includes two items: must be met and added value. Among them, must be met includes: delay increase ≥ 40% or resource utilization decrease ≥ 35%, and added value includes: exposing unrecorded system vulnerabilities, such as communication conflicts in specific hardware combinations.

[0113] Model adversarial training involves mapping the perturbation parameters and performance degradation in negative samples into training labels for the ternary graph model. If a GPU node experiences a 25% increase in task failure rate when its frequency is reduced by 20%, the node's "computational reliability" dimension in the vector is reduced. If an InfiniBand path experiences a 300% increase in transmission latency at 5% packet loss, the weight base of its communication edges is permanently increased by 50%. The trained model parameters are synchronized to the real-time validation layer in real time, and candidate weight generation solutions reinforced with negative samples are prioritized in sandbox testing.

[0114] Among them, the abnormal working condition response mechanism includes disturbance intensity out-of-control processing, negative sample overfitting protection and real-time production environment protection;

[0115] Handling of out-of-control disturbance intensity, including: When a single injection causes system performance degradation of >80%, such as the entire task chain crashes, the engine automatically rolls back to the previous stable state and splits the disturbance factors: network congestion and hardware frequency reduction are applied in batches to locate the specific failure cause. If it is determined to be a specific hardware compatibility issue, such as an abnormal driver after AMDGPU frequency reduction, the hardware model will be added to the avoidance list. Among them, the disturbance segmentation strategy includes:

[0116] First-level segmentation: separating network congestion from hardware frequency reduction;

[0117] Second level segmentation: frequency reduction is applied based on hardware model classification;

[0118] The third level of segmentation: The same hardware model is tested in batches with different frequency reduction ratios.

[0119] Negative sample overfitting protection includes: When 10 consecutive negative samples are caused by the same type of perturbation, such as NVLink path loss, the engine actively introduces orthogonal perturbation types, such as CPU cache pollution simulation, to force the model to learn generalization capabilities. At the same time, 5% of the original model parameters are retained, and hot swapping is performed when the failure rate of the new parameters in the validation layer test exceeds 15%. The overfitting protection trigger conditions are as follows:

[0120] The same type of perturbation continuously generates ≥10 valid negative samples;

[0121] The failure rate of the new model parameters in the first round of testing at the validation layer is >12%.

[0122] Real-time production environment protection includes: isolating physical resources during training, using containerized resource pools to simulate hardware downtime, and prohibiting direct access to production equipment. Data stream transmission perturbations are implemented through software-defined networking, with no actual hardware performance loss. The model parameter update rules are as follows:

[0123] The communication edge weight update amount = original weight × (1 + actual delay increase / 100), and the hardware node vector decay rate = MIN (50%, performance decay rate × 0.8).

[0124] A method for optimizing AI server data processing based on distributed heterogeneous computing, comprising the following steps:

[0125] S1: Constructing a hardware-data-task ternary graph model;

[0126] S2: Generate a joint scheduling and data prefetching scheme through a spatiotemporal cooperative propagation algorithm;

[0127] S3: Perform two-layer closed-loop verification: real-time sandbox testing and negative sample adversarial training.

[0128] It is important to further clarify that during the implementation phase, a hardware-data-task ternary graph model is constructed based on the physical topology of the current distributed cluster. The model scans all online compute nodes, creates a hardware node for each activated GPU / FPGA unit, and injects a real-time monitoring vector into it. The real-time monitoring vectors include three dimensions: computing power availability, memory efficiency, and network affinity. The distributed storage system is traversed to generate data nodes for data shards stored across nodes, annotated with location cost and access popularity. The input AI task stream is parsed, and atomic tasks are mapped to task nodes, labeling the computational mode and dependency depth. The three types of nodes are connected via dynamically weighted directed edges. Communication edge weights between hardware nodes are initialized based on the actual network link type: NVLink = 0.3, RDMA = 0.7, and Ethernet = 1.2. Dependency edge weights between task nodes are set based on the strength of data dependencies: strong dependencies = 1.0 and weak dependencies = 0.3. Initial weights are bound between tasks and hardware based on historical execution efficiency.

[0129] When executing the spatiotemporal collaborative propagation algorithm to generate a joint solution: the forward propagation phase advances from the root node of the task graph. When predicting the start time of downstream tasks, if the hardware node binding edge weight is found to be less than 0.4, the inefficient device is identified and an alternative hardware evaluation branch is automatically inserted; the reverse propagation phase monitors the data flow readiness. When it is detected that the GPU queue idle rate is greater than 25% and the data prefetch path weight is greater than 1.0, the path switching decision tree is started: the same-node memory copy is enabled first, the shared channel within the NUMA domain is second, and cross-node compressed transmission is enabled last. If multiple tasks compete for the same high-speed link, arbitration is performed according to the formula: priority = task depth × delay impact factor. The weight increases by 2 times for each level of depth increase. Among them, the binding edge weight update rules include:

[0130] First execution of a task: Weight = 1 / average time of similar tasks;

[0131] Subsequent update: New weight = 0.6 × old weight + 0.4 × (1 / time taken this time).

[0132] The two-layer closed-loop verification operates according to the following logic:

[0133] During the real-time sandbox testing phase, the production environment topology is cloned, desensitized data streams are injected, and candidate solutions are executed in parallel. If the actual hardware idle rate deviates from the predicted value by more than 5% for a continuous period, the solution is automatically discarded and the failure mode is recorded. If the data arrival delay exceeds the preset value of 10 milliseconds three times in a row, an emergency correction of the path weight is triggered.

[0134] During the negative sample adversarial training phase, the optimal solution trajectory is captured and two types of perturbations are injected: 5% to 15% random packet loss is imposed on the RDMA path, or a 20% to 50% frequency reduction is simulated on the bound GPU. When the perturbation causes the task delay to increase by more than 40%, features are extracted to update the ternary graph model, namely: the "computational reliability" vector dimension of the frequency-reduced hardware node is proportionally reduced according to the decay rate, and the communication edge weight base of the high packet loss path is permanently increased by 30% to 50%. The parameters obtained from the training are synchronized to the sandbox test environment in real time, forming a self-reinforcing loop. Among them, the termination conditions of negative sample training include:

[0135] Success scenario: The new model solution achieves a latency increase of ≤25% under 10% packet loss and 30% frequency reduction.

[0136] Failure scenario: The model's anti-interference ability is not improved after 5 consecutive training sessions.

[0137] During the response to abnormal working conditions, if a hardware node suddenly encounters a failure during model construction, such as an FPGA temperature alarm, its binding edges are immediately frozen and an alternative node is started: the available device with the second highest efficiency for the same type of task is retrieved from the historical execution records; when back propagation detects that all paths of the key data shard are degraded, that is, the weight is greater than 1.5, and compressed transmission is not feasible, that is, the data entropy is greater than 0.9, the local copy regeneration is triggered: the free storage space of the target node is seized to rebuild the data, and a temporary data node is created to join the graph model;

[0138] If three consecutive solutions fail due to the same reason during sandbox testing, such as NVLink congestion, the feedback verification layer will specifically strengthen the disturbance intensity of the scenario, forcing the new model to generate an anti-disturbance solution.

[0139] The path switching threshold includes the following three conditions:

[0140] Same-node switching condition: current path weight > 0.0;

[0141] NUMA domain switching: The current path weight is greater than 0.4 and the path within the domain is available;

[0142] Compressed transmission is enabled: the current path weight is greater than 0.8 and the data compressibility rate is greater than 40%.

[0143] The combined solution reduces the error between compute unit wakeup time and data arrival time to less than 5 milliseconds, and reduces cross-node data migration by over 40% compared to the unoptimized solution. It should be noted that during implementation, the system adhered to two core performance metrics when implementing the combined optimization solution: the error between compute unit wakeup time and dependent data arrival time was kept within 5 milliseconds, and the total cross-node data migration volume was reduced by over 40% compared to the baseline solution. The first metric was achieved using a four-level time synchronization mechanism: when a task is scheduled to the target hardware unit, the system predicts the data arrival time based on the hardware communication edge weights. If the GPU node needs to receive 10GB of intermediate data from the FPGA node and the RDMA path weight indicates a transmission time of 48 milliseconds, the hardware wakeup command is issued 53 milliseconds before the data arrives. If backpropagation detects a risk of prediction deviation due to network jitter, such as a sudden increase in packet loss causing actual transmission delays to exceed 60 milliseconds, emergency channel switching is immediately initiated: direct memory copy within the compute node is prioritized, followed by shared links within the NUMA domain, and finally, data sharding and parallel transmission. If all paths cannot meet the time window requirements, the data accuracy is forced to be reduced to reduce the transmission volume by 35% to compress the time.

[0144] Data migration reduction is ensured through a three-pronged strategy. First, during the task dependency resolution phase, reusable data shards are identified. When three consecutive convolutional tasks require the same set of weights, a shared replica is created at the first task's execution location and its storage location is locked. Subsequent tasks are then forcibly scheduled to that node to avoid migration. Second, intelligent compression decisions are implemented in the data prefetch path selection. A dynamic lossy compression algorithm is used for floating-point intermediate data, with a compression threshold set to 45%. The ZFP compression module is automatically activated when backpropagation determines that the feature map data entropy is less than 0.7. Finally, the storage topology optimizes replica distribution, dynamically adjusting the number of replicas based on the data node access heat vector. Specifically, data with heat > 0.8 is replicated across three cabinets, while data with heat < 0.3 retains a single replica. When a dataset's cross-node migration volume is detected to have increased by more than 15% year-over-year, a simulated perturbation is injected into the feedback validation layer: the data is falsely labeled as having a high heat state, inducing the system to create redundant replicas. The heat determination model is then corrected through negative training.

[0145] A degradation strategy is implemented under abnormal operating conditions to ensure performance. When global network congestion causes the time error to approach the 5-millisecond threshold, a data truncation mechanism is activated for non-critical tasks (i.e., priority ≤ 6), such as discarding 10% of pixels from the edge of an image. If the amount of data migration is forced to increase due to a sudden failure, such as a primary replica node downtime, the system automatically triggers a compensation mechanism after the migration is complete. This mechanism involves merging an equal amount of low-frequency cold data into storage to free up storage space and ensure a balanced total migration volume within the cycle. If the 40% migration reduction target is not achieved for three consecutive task cycles, the dynamic collaborative scheduling subsystem removes local optimal constraints, allowing some tasks to be migrated to more distant nodes with double the computational efficiency. This indirectly reduces the global migration load by reducing the total number of task execution rounds.

[0146] Parameter updates generated by negative adversarial training directly drive the iteration of the ternary graph model's vector generation strategy. It should be noted that, in specific implementations, these updates directly drive the iteration of the ternary graph model's vector generation strategy, forming a closed-loop self-evolutionary mechanism. After the feedback validation layer completes a round of adversarial training, the new parameter package is injected into the model engine through a versioned channel. First, a difference threshold is compared with the current production model. If the average adjustment in the "computing reliability" dimension of the hardware node vectors exceeds 15% or the cardinality of communication edge weights changes by more than 20%, a hot update process is triggered. The hot update is only effective if the latency fluctuation of critical tasks in sandbox testing is ≤18ms and the migration reduction is ≥37%. During the update, a dual-version parallel mode is used: the new parameters undergo three rounds of stress testing in the sandbox environment, injecting 8%-12% packet loss and 25%-35% hardware frequency reduction, while the old version maintains production scheduling. The switchover is completed only if the new parameter model meets the following conditions during testing: critical task latency fluctuation is less than 18ms and the data migration reduction is stabilized within the range of 37%-42%.

[0147] The parameter iteration process incorporates built-in conflict resolution logic: If a GPU node's computational reliability vector is detected to have been downgraded by 30% by negative training, but real-time monitoring shows that the node has actually had zero failures over the past 24 hours, the system automatically initiates a false positive check. Specifically, the system replicates the node's frequency reduction scenario ten times in a sandbox. If the median performance degradation is less than 70% of the training report value, the parameter update is frozen and marked as an overfitting example. If multiple sets of parameter updates are mutually exclusive, such as requiring an increase in communication edge weights while the data node location vector recommends reducing transmission requirements, a decision tree is used for arbitration: the parameter combination that maximizes end-to-end latency reduction is prioritized; if latency impacts are comparable, the solution that reduces cross-node transmission is selected.

[0148] Version rollback under abnormal conditions: If two or more failure modes occur in the production environment after new parameter deployment, such as concurrent GPU task queue timeouts and RDMA checksum errors, and root cause analysis points to parameter updates, the system immediately rolls back to the previous stable version. A fault analysis report is automatically generated after the rollback: hardware state and data flow snapshots at the time of the failure are extracted, the failure scenario is reconstructed in the feedback verification layer, and the problematic parameter is located using the control variable method. Three consecutive rollback events trigger a conservative iteration strategy: subsequent update packages are split into atomic parameter groups, such as updating only network-related edge weights. These groups are then merged and deployed after passing verification. If model evolution stagnates due to long-term conservative iteration, such as when the update amplitude is less than 5% for 20 consecutive rounds, high-risk, high-gain parameters are forcibly injected, such as a ±40% perturbation of the core switch path weights, to break out of the local optimal trap. The forced breakthrough mechanism is activated when the model evolution stagnates for more than 20 rounds or when there is no performance improvement in the production environment for three months. The version rollback threshold is: ≥2 related failure events occur within two hours of new parameter deployment.

[0149] By constructing a hardware-data-task ternary graph model and a spatiotemporal collaborative propagation algorithm, deep collaborative optimization of computing resource scheduling and data stream transmission is achieved. Through the dynamic collaborative scheduling subsystem, based on real-time resource monitoring and task dependency analysis, computing nodes, data storage, and processing tasks are abstracted into graph nodes. Hardware communications, task dependencies, and binding relationships are mapped through weighted directed edges. Forward propagation is used to generate the initial scheduling plan, and backpropagation is used to dynamically adjust the data prefetch path. A tearing rescheduling mechanism is combined to locally optimize critical path delays. This mechanism controls the error between the wake-up time of the computing unit and the arrival time of the data, reduces the amount of data migration across nodes, improves the utilization of heterogeneous resources, and avoids hardware idleness caused by data dependency waiting or cross-node transmission delays.

[0150] A self-reinforcing feedback mechanism is formed through real-time sandbox testing and negative sample adversarial training. The real-time verification layer injects shadow data streams in an isolated environment to test candidate solutions in parallel, ensuring the feasibility of the scheduling solution by comparing hardware idle rates with data delay predictions. The feedback verification layer injects disturbances such as simulated network congestion and hardware frequency reduction based on the historical optimal solution, uses the generated negative sample data to train the ternary graph model, and dynamically updates parameters such as hardware vectors and communication edge weights to improve the system's robustness to abnormal operating conditions. This "test-train-iterate" closed-loop mechanism enables the system to adapt to dynamic load changes and sudden failures in distributed heterogeneous environments, continuously optimize scheduling strategies, and form a complete intelligent optimization chain from data perception, collaborative scheduling, to verification feedback, providing an excellent solution for high-throughput, low-latency AI data processing needs.

[0151] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0152] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An AI server data processing optimization system based on distributed heterogeneous computing, comprising: Distributed heterogeneous computing cluster, dynamic collaborative scheduling subsystem and two-layer closed-loop verification subsystem; features: The distributed heterogeneous computing cluster includes a plurality of interconnected computing nodes, each node integrating at least two hardware units; The dynamic collaborative scheduling subsystem is connected to the heterogeneous resource monitoring module and the task dependency parsing module to construct a hardware-data-task ternary graph model and perform collaborative scheduling; The two-layer closed-loop verification subsystem is connected to the dynamic collaborative scheduling subsystem and includes a real-time verification layer and a feedback verification layer; The hardware-data-task ternary graph model is constructed by abstracting computing nodes, storage locations, and data processing tasks into three types of graph nodes. A vector containing real-time computing power and communication latency is dynamically generated for hardware nodes; a vector containing storage location and access popularity is generated for data nodes; and a vector containing computing mode and dependency relationships is generated for task nodes. Hardware communication links, task data dependencies, and task hardware binding relationships are mapped into weighted directed edges. The edge weights between task nodes and hardware nodes reflect the expected execution efficiency of the task on the hardware. The collaborative scheduling includes a spatiotemporal collaborative communication algorithm: Forward propagation phase: Starting from the root task node with no dependencies, the earliest start time of each subtask is predicted layer by layer along the task dependency edges, and an initial scheduling plan is generated based on the execution efficiency of the bound hardware nodes. Backward propagation phase: Backtracking from the last-level task node, if it is detected that the hardware node is idle due to unready data, the data source node is traced along the hardware communication edge, and the data prefetching time window is dynamically shortened or the prefetching path is switched; During the construction of the hardware-data-task ternary graph model, a tearing rescheduling mechanism is used to handle task delays. This includes: when the critical path task delay exceeds a threshold, only the associated subgraph is deconstructed; the spatiotemporal collaborative propagation algorithm is re-executed within the subgraph range to output a local optimization solution; The real-time verification layer performs the following operations: injecting shadow data streams into the sandbox environment to test candidate solutions in parallel, comparing the difference between the actual and predicted hardware idle rates, and comparing the difference between the actual and predicted data arrival delays; The feedback verification layer captures the optimal execution trajectory of the production environment, injects simulated network congestion and hardware frequency reduction disturbances through the negative sample adversarial training engine; uses the perturbation data to train the ternary graph model, and updates the model parameters to the real-time verification layer.

2. The AI ​​server data processing optimization system based on distributed heterogeneous computing according to claim 1, characterized in that: The heterogeneous resource monitoring module collects the computing power utilization, memory bandwidth occupancy and cross-device communication delay of the hardware unit in real time; the task dependency parsing module dynamically generates a directed acyclic graph of AI data processing tasks and identifies data dependencies.

3. A method for optimizing AI server data processing based on distributed heterogeneous computing, applied to the system according to any one of claims 1 to 2, characterized in that: The steps include: S1: Constructing a hardware-data-task ternary graph model; S2: Generate a joint scheduling and data prefetching scheme through a spatiotemporal cooperative propagation algorithm; S3: Perform two-layer closed-loop verification: real-time sandbox testing and negative sample adversarial training.

4. The method for optimizing AI server data processing based on distributed heterogeneous computing according to claim 3, characterized in that: The combined solution makes the error between the computing unit wake-up time and the data arrival time less than a predetermined time threshold; and the cross-node data migration amount is reduced by more than a predetermined ratio threshold compared with the non-optimized solution.

5. The method for optimizing AI server data processing based on distributed heterogeneous computing according to claim 4, characterized in that: The parameter update output by the negative sample adversarial training directly drives the vector generation strategy iteration of the ternary graph model.

Images