An AI computing resource adaptive orchestration system for multi-task scenarios

By generating static spectral feature metadata and monitoring resonance status in real time, the task execution order is adjusted, which solves the problem of micro-architecture resource contention in multi-task concurrency and improves the utilization efficiency of computing resources.

CN123068092APending Publication Date: 2026-11-10HAIXUAN TECHNOLOGY (BEIJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611237349.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-14
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing technologies lack prior awareness of the resource request patterns of the underlying microarchitecture of tasks during concurrent execution of multiple tasks. This leads to overlapping memory access pulses when tasks on the same physical computing node access the underlying bus and cache, causing synchronous competition and bus congestion in the hardware microarchitecture, resulting in a decrease in the throughput of computing resources.

Method used

Static spectral feature metadata is generated through a computation graph semantic parsing and feature injection module. Tasks are deployed using a global orthogonal compatibility scheduling module. Combined with an on-chip micro-spectrum monitoring module and a hardware-level dynamic instruction interleaving control module, the execution order of tasks is monitored and adjusted in real time to avoid resonance and achieve resource optimization at the micro level.

Benefits of technology

It effectively alleviates the competition for micro-architecture resources in multi-tasking scenarios, improves the execution throughput of computing resources, and enables the safe interleaving of tasks without suspending system processes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123068092A_ABST
    Figure CN123068092A_ABST
Patent Text Reader

Abstract

This invention discloses an adaptive orchestration system for AI computing resources in multi-task scenarios. It includes a computation graph semantic parsing and feature injection module, used to convert the memory access request sequence of AI tasks into static spectral feature metadata and inject it into the executable binary file; a global orthogonal compatibility scheduling module, which calculates interference costs based on the static spectral feature metadata and constructs a global interference cost matrix, performing graph partitioning calculations to achieve orthogonal compatible combined deployment of multiple tasks; an on-chip micro-spectrum monitoring module, which reads hardware counter values ​​to extract the dynamic operating spectrum during operation, detects resonance states, and calculates the instantaneous physical phase difference; and a hardware-level dynamic instruction interleaving control module, which injects pause cycle instructions into the hardware transmit queue and, when the duty cycle exceeds the limit, coordinates with the host operating system to perform microkernel operator fission reconstruction on the remaining tensor data region. This invention effectively alleviates bus frequency contention in the underlying microarchitecture.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence computing resource scheduling and microarchitecture performance optimization technology, specifically to an AI computing resource adaptive orchestration system for multi-task scenarios. Background Technology

[0002] In artificial intelligence applications, concurrent execution of multiple tasks is a common operating mode for improving computing power utilization. During concurrent execution, the processor hardware needs to handle concurrent computing tasks and data memory access operations. Existing technologies typically employ operating system-based task scheduling strategies and general software queues for resource allocation to coordinate the resource usage of different computing tasks within physical computing nodes.

[0003] However, existing technologies, when performing macro-level software scheduling, lack the prior awareness of the resource request patterns of the underlying micro-architecture of tasks, remaining only at the macro-level of computing power or memory capacity allocation. For example, existing Kubernetes or Slurm schedulers only perform macro-level allocation based on computing power quotas and memory capacity, without considering the micro-architecture time-frequency characteristics of operator-level memory access requests within tasks. This scheduling method cannot pre-quantify the discrete memory access waveform characteristics within tasks, leading to overlapping memory access pulses when multiple concurrent tasks assigned to the same physical computing node access the underlying bus and cache within the same clock cycle. This, in turn, causes competition and bus congestion in the underlying hardware micro-architecture, resulting in a decrease in the execution throughput of computing resources. Summary of the Invention

[0004] To address the problems mentioned in the background, this invention provides an AI computing resource adaptive orchestration system for multi-task scenarios, comprising: a computation graph semantic parsing and feature injection module, used to receive an AI task to be compiled, generate a memory access request sequence corresponding to the AI ​​task and convert it into static spectrum feature metadata, and inject the static spectrum feature metadata into the executable binary file corresponding to the AI ​​task; a global orthogonality compatibility scheduling module, used to extract the static spectrum feature metadata from the executable binary files of multiple AI tasks to be scheduled to calculate the interference cost, and distribute multiple AI tasks that meet the orthogonality compatibility condition to the same physical computing node; and an on-chip micro-spectrum monitoring module, used to monitor the spectrum of the AI ​​task on the physical computing node. When the node's AI acceleration chip executes AI tasks, it collects dynamic runtime time series, performs hardware-level time-frequency conversion to extract dynamic runtime spectrum, and sends a resonance warning signal with a hardware context identifier when it determines that multiple concurrently executing AI tasks have entered a resonance state. The hardware-level dynamic instruction interleaving control module receives the resonance warning signal, extracts the instantaneous physical phase difference of the AI ​​tasks in the resonance state based on the hardware context identifier, and injects pause cycle instructions into the hardware launch queue corresponding to non-critical tasks. When the cumulative number of pause cycle instructions reaches the hardware tolerance upper limit threshold, it sends a hardware interrupt signal to the host operating system to trigger the microkernel operator fission reconstruction of the target tensor operator.

[0005] Preferably, when the computation graph semantic parsing and feature injection module generates the memory access request sequence, it is specifically used for: extracting the directed acyclic graph structure corresponding to the artificial intelligence task, traversing the nodes in the directed acyclic graph structure to identify the tensor operator type, and dividing the nodes into computationally intensive operators and memory-intensive operators; retrieving the hardware parameter configuration information of the target hardware platform according to the distribution sequence of computationally intensive operators and memory-intensive operators, and configuring the microarchitecture simulator based on the hardware parameter configuration information; inputting the tensor operators in the directed acyclic graph structure into the microarchitecture simulator according to the topological sorting rule for static deduction, and outputting the ideal memory access request rate of each tensor operator; according to the time axis set by the topological sorting, sequentially splicing the ideal memory access request rates of tensor operators with dependencies, and time-domain superimposing the ideal memory access request rates of parallel tensor operators without dependencies, generating a discrete-time sequence signal that varies with the processor clock cycle as the memory access request sequence.

[0006] Preferably, when the computation graph semantic parsing and feature injection module generates and injects static spectral feature metadata, it is specifically used to: pad the length of the memory access request sequence with zeros to a power of 2, perform a discrete Fourier transform operation to convert it into a frequency domain signal, sort it in descending order according to the amplitude value, extract the frequency components whose sorting position is less than or equal to a preset extraction dimension threshold as feature frequencies and their corresponding amplitude values, and construct static spectral feature metadata; serialize and encode the static spectral feature metadata using a continuous key-value pair arrangement format to generate a binary data stream; allocate an independent custom data storage segment inside the executable binary file to write the binary data stream, and record the starting memory offset address and effective data length of the custom data storage segment in the file header structure information.

[0007] Preferably, when the global orthogonal compatibility scheduling module calculates the interference cost, it specifically performs the following steps: reads the file header structure information of the executable binary file, extracts the static spectral feature metadata based on the starting memory offset address and effective data length recorded in the file header structure information, performs deserialization decoding to obtain the feature frequency values ​​and amplitude value sets corresponding to each scheduled artificial intelligence task; iterates and combines to select any two scheduled artificial intelligence tasks as the first task and the second task respectively, and calculates the interference cost (S) of the first task and the second task in the frequency domain dimension based on the following formula:

[0008]

[0009] in, Indicates the cost of interference; and These represent the total number of feature frequencies contained in the first task and the second task, respectively. and These represent the amplitude and characteristic frequency values ​​of the first task, respectively. and These represent the amplitude and characteristic frequency values ​​of the second task, respectively. This represents a constant correction factor.

[0010] When the global orthogonal compatibility scheduling module performs deployment, it specifically performs the following: Constructs a global interference cost matrix based on the pairwise interference costs between multiple AI tasks to be scheduled; updates interference cost values ​​greater than or equal to a preset frequency domain orthogonality threshold to preset maximum constants; and uses interference cost values ​​less than the frequency domain orthogonality threshold as orthogonal compatible connection edges; maps the updated global interference cost matrix to an undirected weighted graph, where the AI ​​tasks to be scheduled are mapped as vertices, the orthogonal compatible connection edges are mapped as edges, and the corresponding interference cost values ​​are mapped as edge weights; obtains hardware resource capacity constraint information of physical computing nodes; performs minimum weight graph partitioning calculation on the undirected weighted graph using the hardware resource capacity constraint information as boundary constraints to obtain a vertex partition set; and deploys multiple AI tasks partitioned into the same set to the same physical computing node based on the vertex partition set.

[0011] Preferably, when the on-chip micro-spectrum monitoring module collects dynamic running time series, it is specifically used to: obtain the hardware context identifier corresponding to the artificial intelligence task, initialize the on-chip hardware performance counter according to the hardware context identifier, and bind its sampling trigger clock source to the hardware microsecond timer;

[0012] Within each trigger cycle of the hardware microsecond timer, event filtering is performed based on the hardware context identifier. The hardware count values ​​in the last-level cache miss event register and the system bus transaction byte register are directly read, and the last-level cache miss rate and system bus bandwidth usage are calculated. Normalization processing is performed according to the preset last-level cache miss rate upper limit threshold and the bus peak bandwidth value. The normalized last-level cache miss rate and system bus bandwidth usage are weighted and summed to obtain the single-point instantaneous memory access load value. The single-point instantaneous memory access load values ​​in multiple consecutive trigger cycles are concatenated into the dynamic running time sequence according to the order of trigger time.

[0013] Preferably, when the on-chip micro-spectrum monitoring module sends a resonance warning signal, it is specifically used for: inputting the dynamic operating time series to the output frequency domain complex signal of the fixed fast Fourier transform logic circuit; using the on-chip hardware computing unit to perform a square root operation on the real and imaginary part values ​​to extract the operating amplitude; extracting frequency components with operating amplitudes greater than a preset amplitude boundary value to construct a dominant operating frequency set; obtaining a standard discrete frequency interval sequence; aligning and mapping the frequency components in the dominant operating frequency set and the feature frequency values ​​in the static spectral feature metadata to the standard discrete frequency interval sequence; generating a dynamic operating spectrum vector and a static feature reference vector and calculating the cosine similarity of the two vectors to obtain a spectrum similarity value; when the spectrum similarity value is lower than a preset drift threshold, writing the dominant operating frequency set into a global shared state register array; reading the global shared state register array to obtain the dominant operating frequency values ​​of other concurrently executed artificial intelligence tasks; and determining that the state of resonance has entered a state of resonance and sending a warning signal when the absolute value of the difference between the dominant operating frequency values ​​of any two artificial intelligence tasks is less than a preset co-frequency tolerance value.

[0014] Preferably, the hardware-level dynamic instruction interleaving control module is specifically used for: obtaining the most recent memory access bandwidth peak timestamp of the first artificial intelligence task and the second artificial intelligence task; combining the physical cycle span corresponding to the dominant operating frequency; mapping the time difference between the current clock time and the most recent memory access bandwidth peak timestamp into a phase angle; extracting the instantaneous physical phase value; and then calculating the absolute value of the difference to obtain the instantaneous physical phase difference; based on the safety inversion threshold, the instantaneous physical phase difference, the dominant operating frequency value, and the core clock frequency value of the artificial intelligence acceleration chip, calculating the number of pause ticks by rounding up using a preset calculation formula; setting the artificial intelligence tasks with lower system priority as non-critical tasks; injecting no-operation instructions equal to the number of pause ticks into the hardware transmit queue corresponding to the non-critical tasks; or lowering the transmit enable level signal of the hardware transmit queue corresponding to the non-critical tasks and maintaining a clock cycle equal to the number of pause ticks.

[0015] Preferably, the system further includes a host operating system; the hardware-level dynamic instruction interleaving control module is further configured to: count the cumulative number of pauses continuously injected into the hardware delivery queue within a preset time window; and send a hardware interrupt signal to the host operating system when the cumulative number of pauses is greater than or equal to the hardware tolerance upper limit threshold; the host operating system is configured to: receive the hardware interrupt signal and suspend the device flow execution queue corresponding to the non-critical task, intercept the target tensor operator in the execution state, record the current memory read pointer and calculation progress status information, and determine the remaining tensor data region that has not yet been executed; the host operating system is further configured to: re-execute the slicing operation on the remaining tensor data region in the spatial dimension to generate multiple reduced-size microkernel computing blocks, and equivalently decompose the target tensor operator into multiple sub-operator sequences and re-distribute them to the artificial intelligence acceleration chip of the physical computing node for continued execution.

[0016] Preferably, the system further includes: an operator branch scheduling module, used to read the precision channel allocation ratio, divide the input feature tensor into a first feature tensor quantum set and a second feature tensor quantum set along the channel dimension, and allocate high-precision floating-point operator branches and low-precision fixed-point operator branches respectively; an asymmetric precision execution module, used to distribute the calculation instruction sequences of the high-precision floating-point operator branches and the low-precision fixed-point operator branches to the floating-point arithmetic unit and the integer arithmetic unit respectively for processing within the same clock cycle, and write the data branch that is calculated first into a hardware synchronization cache queue to wait until both output an end signal; and a precision alignment and feature fusion module, used to, after receiving the end signal, perform dequantization mapping on the low-precision feature tensor result based on the quantization scaling factor and the numerical offset to obtain a format-aligned feature tensor, and perform matrix concatenation operation on the high-precision feature tensor result and the format-aligned feature tensor in the channel dimension to obtain a fused feature tensor.

[0017] The present invention, by adopting the above technical solution, can bring the following beneficial effects:

[0018] 1. This invention converts the memory access request sequence of artificial intelligence tasks into frequency domain signals and extracts static spectral feature metadata, which is then encapsulated into the header structure information of an executable binary file. Interference costs are then calculated based on the characteristic frequency and amplitude values. In addition, an undirected weighted graph is constructed by combining the frequency domain orthogonality threshold and hardware capacity constraints to perform minimum weight graph partitioning calculation. This enables the orthogonal and compatible combination deployment of multiple artificial intelligence tasks at the cluster level and effectively alleviates the problem of competition for resources at the same frequency in the underlying microarchitecture.

[0019] 2. This invention constructs a dynamic sequence by reading hardware count values ​​within the microsecond timer trigger cycle and extracting the spectrum using a fixed circuit to send a synchronous resonance warning signal. It maps the time difference to a phase angle to obtain the instantaneous physical phase difference and calculates the number of pauses based on this. Then, it injects no-operation instructions or pulls down the transmit enable signal into the hardware transmit queue of non-critical tasks, thereby delaying the transmission of underlying instructions without suspending system processes, so as to ensure that multi-task memory access pulses are safely interleaved on the time axis.

[0020] 3. This invention determines the memory access duty cycle out of bounds by statistically analyzing the cumulative number of pauses during continuous injection and sends an interrupt signal to suspend the execution queue when the upper limit is triggered. It intercepts the target tensor operator and determines the remaining tensor data region based on the current memory read pointer and progress status. Then, it re-slices the remaining region to generate microkernel blocks and re-issues them for execution. This achieves the physical change of the temporal distribution of task memory access requests to relieve the synchronous resonance blocking state that cannot be repaired by hardware delay instructions alone. Attached Figure Description

[0021] Figure 1 This is a diagram of the AI ​​computing power resource adaptive orchestration system architecture in a multi-task scenario according to the present invention.

[0022] Figure 2 This is a flowchart of the AI ​​computing power resource adaptive orchestration method in a multi-task scenario according to the present invention;

[0023] Figure 3 This is a schematic diagram of the processing logic of the computational graph semantic parsing and feature injection module of the present invention;

[0024] Figure 4 This is a flowchart of the frequency domain conversion and static signature feature metadata injection process of the present invention;

[0025] Figure 5 This is a flowchart illustrating the construction of the multi-task spectrum interference cost function of this invention.

[0026] Figure 6 This is a flowchart of the global cost matrix solution and orthogonal compatibility node placement for this invention.

[0027] Figure 7 This is a flowchart of the real-time acquisition process of microsecond-level time-domain signals based on HPM according to the present invention;

[0028] Figure 8 This is a flowchart of the on-chip hardware frequency domain conversion, spectrum drift, and resonance early warning detection of the present invention;

[0029] Figure 9 This is a flowchart of the nanosecond-level phase difference calculation and instruction pause adaptive injection process of the present invention;

[0030] Figure 10This is a flowchart of the multi-task duty cycle out-of-bounds detection and dynamic operator fission of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Example 1

[0033] Please refer to Figure 1 The present invention provides an AI computing power resource adaptive orchestration system for multi-task scenarios, including: a computation graph semantic parsing and feature injection module 101, a global orthogonal compatibility scheduling module 102, an on-chip micro-spectrum monitoring module 103, and a hardware-level dynamic instruction interleaving control module 104.

[0034] The computation graph semantic parsing and feature injection module 101 is deployed in the system's software framework layer. The computation graph semantic parsing and feature injection module 101 is used to parse the directed acyclic graph structure of the artificial intelligence task during the task compilation stage, obtain the memory access request sequence of each operator inside the task, perform discrete Fourier transform on the memory access request sequence to extract feature frequencies and corresponding amplitudes, and then generate static spectral feature metadata. The static spectral feature metadata is written into the data segment of the executable binary file. Through the pre-generation and injection of static spectral feature metadata, the AI ​​computing power resource adaptive orchestration system in multi-task scenarios can grasp the micro-architecture resource access rules of the artificial intelligence task before the task runs, and reduce the computing power analysis overhead during dynamic runtime.

[0035] The global orthogonal compatibility scheduling module 102 is deployed on the cluster scheduling node. The global orthogonal compatibility scheduling module 102 is used to receive scheduling requests from multiple artificial intelligence tasks, extract static spectrum feature metadata from the corresponding executable binary file, calculate the interference cost of different artificial intelligence tasks in the frequency domain dimension, construct a cost matrix based on the interference cost, and then allocate multiple tasks with interference costs lower than the preset orthogonal threshold to the same physical computing node. Through orthogonal matching and node allocation in the frequency domain dimension, the clustering of tasks with similar memory access characteristics is avoided from the macro scheduling level, and the probability of bus contention within the physical computing node is reduced.

[0036] The on-chip micro-spectrum monitoring module 103 is integrated inside the artificial intelligence acceleration chip of the physical computing node. The on-chip micro-spectrum monitoring module 103 runs independently of the host operating system kernel. The on-chip micro-spectrum monitoring module 103 parses the executable binary file and extracts static spectrum feature metadata. It collects timing data of the last-level cache miss rate and system bus bandwidth usage through the on-chip hardware performance counter. It uses on-chip logic circuits to perform time-frequency conversion on the timing data to obtain the dynamic running spectrum. It compares the dynamic running spectrum with the static spectrum feature metadata to detect the resonance state between tasks on the same node. By directly performing time-frequency conversion and state detection through the hardware performance counter and on-chip logic circuits, it avoids interruption communication delays caused by the involvement of the host operating system and achieves microsecond-level micro-architecture state awareness.

[0037] The hardware-level dynamic instruction interleaving control module 104 is deployed in the instruction scheduling logic unit of the artificial intelligence acceleration chip. When the on-chip micro-spectrum monitoring module 103 detects a resonance state, the hardware-level dynamic instruction interleaving control module 104 obtains the instantaneous physical phase difference of multiple tasks that are resonating. When the instantaneous physical phase difference is less than the set safety threshold, the hardware-level dynamic instruction interleaving control module 104 injects pause cycles into the launch queue of non-critical tasks. When the number of injected pause cycles reaches the set upper limit threshold, the hardware-level dynamic instruction interleaving control module 104 sends a control signal to the host to trigger operator fission operation, adjusts the computing block size of the corresponding task on the physical computing node, realizes the hardware-level instruction staggered launch by injecting pause cycles into the launch queue, and adjusts the physical frequency by operator fission when the pause adjustment range is exceeded, so as to maintain the bus throughput in the multi-task concurrent scenario.

[0038] Among the above modules, the computation graph semantic parsing and feature injection module 101 establishes a data connection with the global orthogonal compatibility scheduling module 102 to transmit static spectrum feature metadata; the global orthogonal compatibility scheduling module 102 communicates with the on-chip micro spectrum monitoring module 103 to issue scheduling instructions containing static spectrum feature metadata to the on-chip micro spectrum monitoring module 103; the on-chip micro spectrum monitoring module 103 establishes an electrical connection with the hardware-level dynamic instruction interleaving control module 104 to transmit the detected co-frequency resonance state and hardware trigger signal to the hardware-level dynamic instruction interleaving control module 104. Thus, the AI ​​computing power resource adaptive orchestration system in multi-task scenarios establishes a cross-stack data interaction path between the software compilation framework and the hardware control logic, solving the information silo problem between computing power network scheduling instructions and chip microarchitecture competition state.

[0039] Example 2

[0040] This embodiment is combined with the appendix Figure 2The main process of the AI ​​computing power resource adaptive orchestration method in multi-task scenarios is described. This main process is executed collaboratively by the modules described in Example 1, forming a complete closed loop from static feature injection in the compilation stage, macro-schedule orthogonal allocation, runtime micro-architecture state monitoring to hardware-level dynamic instruction interleaving control.

[0041] Please refer to Figure 2 This invention provides an adaptive orchestration method for AI computing resources in a multi-task scenario, comprising the following steps:

[0042] In step S100, the computation graph semantic parsing and feature injection module 101 acquires the artificial intelligence task to be processed and parses the directed acyclic graph structure of the artificial intelligence task during the software compilation stage. The computation graph semantic parsing and feature injection module 101 extracts the memory access request sequence of each operator inside the artificial intelligence task through static architecture simulation, performs discrete Fourier transform on the memory access request sequence to obtain static spectral feature metadata containing feature frequencies and corresponding amplitudes. The computation graph semantic parsing and feature injection module 101 injects the static spectral feature metadata into the executable binary file corresponding to the artificial intelligence task. By converting the time domain features into frequency domain metadata and binding them to the binary file during the compilation stage, the scheduling node can obtain the underlying memory access behavior pattern of the artificial intelligence task without performing dynamic testing before the task runs.

[0043] In step S200, the global orthogonal compatibility scheduling module 102 receives scheduling requests from multiple artificial intelligence tasks; the global orthogonal compatibility scheduling module 102 extracts static spectrum feature metadata from multiple executable binary files, and calculates the interference cost of multiple artificial intelligence tasks in the frequency domain dimension based on the static spectrum feature metadata; the global orthogonal compatibility scheduling module 102 constructs a global cost matrix based on the interference cost; the global orthogonal compatibility scheduling module 102 allocates artificial intelligence tasks with interference costs lower than a preset orthogonal threshold in the global cost matrix to the same physical computing node, and schedules artificial intelligence tasks with complementary memory access frequencies to the same computing environment through cost calculation and orthogonal matching allocation in the frequency domain dimension, thereby reducing the probability of physical collisions on the system bus from a macroscopic perspective;

[0044] In step S300, during the execution phase of the artificial intelligence task on the physical computing node, the on-chip micro-spectrum monitoring module 103 reads the static spectrum feature metadata from the executable binary file; the on-chip micro-spectrum monitoring module 103 collects timing data of the final-level cache miss rate and system bus bandwidth usage through the on-chip hardware performance counter inside the artificial intelligence acceleration chip of the physical computing node; the on-chip micro-spectrum monitoring module 103 uses on-chip logic circuits to perform time-frequency conversion on the timing data to obtain the dynamic running spectrum; the on-chip micro-spectrum monitoring module 103 compares the dynamic running spectrum with the static spectrum feature metadata to determine whether multiple artificial intelligence tasks on the same physical computing node have entered a state of resonance, and by collecting and converting the spectrum in real time through hardware-level performance counters and logic circuits, the communication overhead of the host operating system is bypassed, thereby improving the real-time performance and accuracy of micro-architecture resource status monitoring;

[0045] In step S400, when the on-chip micro-spectrum monitoring module 103 determines that multiple artificial intelligence tasks have entered a state of resonance, the hardware-level dynamic instruction interleaving control module 104 calculates the instantaneous physical phase difference between the multiple artificial intelligence tasks that have entered the state of resonance. When the instantaneous physical phase difference is less than the set safety threshold, the hardware-level dynamic instruction interleaving control module 104 injects a pause period into the preset non-critical task launch queue to stagger the hardware bus access time between multiple artificial intelligence tasks. Through nanosecond-level instruction launch delay injection, hardware-level interleaving and peak shifting of instruction streams are achieved without suspending or migrating artificial intelligence tasks, thus ensuring the throughput of physical computing nodes.

[0046] In step S500, when the number of pause cycles continuously injected by the hardware-level dynamic instruction interleaving control module 104 reaches the set upper limit threshold, the hardware-level dynamic instruction interleaving control module 104 sends a control signal to the host operating system. The host operating system receives the control signal and triggers operator fission operation to adjust the tensor calculation block size of the corresponding artificial intelligence task on the physical computing node, reconstruct the physical execution frequency of the corresponding artificial intelligence task, and actively change the inherent memory access frequency of the artificial intelligence task in the event of phase control failure by setting the upper limit threshold and triggering operator fission, thus forming a closed-loop control mechanism that combines microarchitecture fine-tuning with operator splitting.

[0047] Example 3

[0048] Based on Example 2, this example combines the appendix. Figure 3 and Figure 4 The detailed execution process of step S100 in Embodiment 2 is explained, that is, the computation graph semantic parsing and feature injection module 101 completes the complete data processing link of static spectrum feature metadata generation and injection during the software compilation stage.

[0049] Please refer to Figure 3 The computation graph semantic parsing and feature injection module 101 performs the following steps for computation graph parsing and memory access waveform time-domain simulation:

[0050] Step S101: The computation graph semantic parsing and feature injection module 101 receives the artificial intelligence task to be compiled, extracts the directed acyclic graph structure corresponding to the artificial intelligence task, traverses the nodes in the directed acyclic graph structure, identifies the tensor operator type corresponding to the node, and by extracting the directed acyclic graph structure and identifying the tensor operator type, clarifies the operation dependency relationship and operator attribute of the artificial intelligence task at the logical execution level.

[0051] In step S102, the computation graph semantic parsing and feature injection module 101 divides nodes into computationally intensive operators and memory-intensive operators according to the tensor operator type. Among them, computationally intensive operators include convolution operators and matrix multiplication operators, and memory-intensive operators include pooling operators and normalization operators. The computation graph semantic parsing and feature injection module 101 retrieves the hardware parameter configuration information of the target hardware platform according to the distribution sequence of computationally intensive operators and memory-intensive operators. By distinguishing operator attributes and reading hardware parameter configuration information, it provides a matching benchmark for subsequent evaluation of the underlying resource consumption characteristics of different nodes.

[0052] Step S103: The computation graph semantic parsing and feature injection module 101 configures the microarchitecture simulator based on the hardware parameter configuration information. The tensor operators in the directed acyclic graph structure are input into the microarchitecture simulator according to the topological sorting rules for static deduction. The microarchitecture simulator outputs the ideal memory access request rate of the tensor operators on the target hardware platform. The ideal memory access request rate is obtained through the static deduction of the microarchitecture simulator. The underlying microarchitecture tick of instruction execution is obtained without running the actual physical chip.

[0053] It should be noted that the ideal memory access request rate is a baseline rate that ignores runtime cache dynamic eviction and multi-task interference, and is used to provide a priori frequency domain profile of task memory access behavior during the compilation phase; the actual runtime spectrum drift is dynamically corrected by the on-chip micro spectrum monitoring module.

[0054] In step S104, the computation graph semantic parsing and feature injection module 101 sequentially concatenates the ideal memory access request rates of dependent tensor operators according to the time axis set by the topological sorting; it then performs time-domain superposition of the ideal memory access request rates of undependent parallel tensor operators; based on the processing results of sequential concatenation and time-domain superposition, it generates the memory access request sequence corresponding to the artificial intelligence task; wherein, the memory access request sequence is a discrete time sequence signal that varies with the processor clock cycle. By processing the ideal memory access request rates with and without dependencies, the discrete and parallel operator logic within the artificial intelligence task is mapped into time-domain waveform features with time continuity, thus constructing the basic data input for the subsequent conversion of feature data to the frequency domain dimension;

[0055] After completing the above time-domain waveform feature generation, the computation graph semantic parsing and feature injection module 101 further performs frequency domain transformation and static feature metadata injection. Please refer to... Figure 4 The computation graph semantic parsing and feature injection module 101 performs the following steps for frequency domain transformation and static signature feature metadata injection:

[0056] In step S105, the computation graph semantic parsing and feature injection module 101 obtains the memory access request sequence, and after padding the length of the memory access request sequence with zeros to a power of 2, performs a discrete Fourier transform operation on the memory access request sequence to convert the memory access request sequence in the time domain dimension into a frequency domain signal. The computation graph semantic parsing and feature injection module 101 sorts all frequency components in the frequency domain signal in descending order according to the amplitude value, and extracts the frequency components whose sorting position is less than or equal to the preset extraction dimension threshold as feature frequencies. At the same time, it obtains the amplitude value corresponding to the feature frequency. By performing a discrete Fourier transform operation and extracting the feature frequency and amplitude value, the frequency domain features that dominate the microarchitecture resource request pattern are separated from the discrete time domain waveform signal.

[0057] Step S106: The computation graph semantic parsing and feature injection module 101 constructs static spectral feature metadata based on the feature frequencies and corresponding amplitude values. The construction formula for the static spectral feature metadata is as follows:

[0058]

[0059] In the formula, This represents static spectral feature metadata; K represents the preset extraction dimension threshold; i represents the index number of the frequency component, and i is a positive integer ranging from 1 to K. This represents the characteristic frequency value at index i; Represents the characteristic frequency value The corresponding amplitude values ​​are used to construct static spectrum feature metadata through the above formula, quantifying the hardware resource request behavior of artificial intelligence tasks into a set of definite mathematical features, and providing underlying data support for the allocation of execution nodes of the global orthogonal compatibility scheduling module 102.

[0060] In step S107, the computation graph semantic parsing and feature injection module 101 serializes and encodes the static spectrum feature metadata, converting it into a binary data stream using a continuous key-value pair arrangement format. The computation graph semantic parsing and feature injection module 101 obtains the executable binary file corresponding to the artificial intelligence task, allocates an independent custom data storage segment within the executable binary file, and then writes the binary data stream into the custom data storage segment. Through serialization and encoding and writing into the custom data storage segment, the physical binding and encapsulation of the static spectrum feature metadata and the underlying execution code of the artificial intelligence task are realized, enabling the task scheduling process to have prior awareness of the underlying microarchitecture environment.

[0061] In step S108, the computation graph semantic parsing and feature injection module 101 modifies the header structure information of the executable binary file, recording the starting memory offset address and effective data length of the custom data storage segment in the header structure information; the computation graph semantic parsing and feature injection module 101 outputs the executable binary file encapsulating static spectral feature metadata to the global orthogonal compatibility scheduling module 102. By recording the starting memory offset address and effective data length in the header structure information, the global orthogonal compatibility scheduling module 102 can directly locate and extract the static spectral feature metadata based on the header structure information when loading the executable binary file, avoiding the time overhead caused by full file scanning;

[0062] Example 4

[0063] Based on Example 2, this example combines the appendix. Figure 5 and Figure 6 The detailed execution process of step S200 in Embodiment 2 is explained, namely, the global orthogonal compatibility scheduling module 102 performs a complete processing link of multi-task spectrum interference cost calculation and orthogonal compatibility node placement based on static spectrum feature metadata.

[0064] Please refer to Figure 5 The steps of the global orthogonal compatibility scheduling module 102 in constructing the multi-task spectrum interference cost function include:

[0065] In step S201, the global orthogonal compatibility scheduling module 102 receives multiple AI tasks to be scheduled within the cluster node and reads the file header structure information of the executable binary files corresponding to the multiple AI tasks to be scheduled. Based on the starting memory offset address and effective data length recorded in the file header structure information, the global orthogonal compatibility scheduling module 102 extracts static spectral feature metadata from the custom data storage segment within each AI task to be scheduled. The global orthogonal compatibility scheduling module 102 performs deserialization decoding on the static spectral feature metadata to obtain the set of characteristic frequency values ​​and amplitude values ​​corresponding to each AI task to be scheduled. By directly extracting the static spectral feature metadata based on the file header structure information and performing deserialization decoding, lossless parsing of the underlying frequency domain attributes of the AI ​​tasks at the macroscopic scheduling layer is achieved, providing accurate data input for quantifying interference costs.

[0066] In step S202, the global orthogonal compatibility scheduling module 102 traverses and combines multiple AI tasks to be scheduled, and selects any two AI tasks to be scheduled as the first task and the second task respectively; the global orthogonal compatibility scheduling module 102 calculates the interference cost of the first task and the second task in the frequency domain based on the characteristic frequency value and amplitude value corresponding to the first task and the characteristic frequency value and amplitude value corresponding to the second task.

[0067] The formulas for calculating the interference cost in the frequency domain for the first and second tasks are as follows:

[0068]

[0069] In the formula, This represents the interference cost between the first task p and the second task q in the frequency domain. This represents the total number of feature frequencies contained in the first task p; The index represents the total number of feature frequencies contained in the second task q; the index represents the index number of the feature frequencies of the first task p; and the index represents the index number of the feature frequencies of the second task q. This represents the amplitude value of index i in the first task p; This represents the amplitude value of index j in the second task q; This represents the feature frequency value with index number i in the first task p; This represents the feature frequency value with index number j in the second task q; Represents the characteristic frequency value With characteristic frequency values The absolute difference; This represents a preset constant correction factor, which is used to prevent computational overflow caused by an absolute difference of zero. By combining the amplitude value and the difference in characteristic frequency through the above formula, the interference cost is calculated, and the frequency overlap risk of resource requests from the underlying microarchitecture of multiple artificial intelligence tasks is transformed into a comparable quantitative numerical indicator, thus establishing a mathematical evaluation criterion for the construction of the global cost matrix.

[0070] Furthermore, after completing the interference cost calculation, the global orthogonal compatibility scheduling module 102 further performs global cost matrix solving and orthogonal compatibility node placement. Please refer to [link / reference needed]. Figure 6 The steps of the global orthogonal compatibility scheduling module 102 in performing global cost matrix solving and orthogonal compatibility node placement include:

[0071] In step S203, the global orthogonal compatibility scheduling module 102 constructs a global interference cost matrix based on the pairwise interference costs between multiple AI tasks to be scheduled; the global orthogonal compatibility scheduling module 102 obtains a preset frequency domain orthogonality threshold; the global orthogonal compatibility scheduling module 102 updates the interference cost values ​​in the global interference cost matrix that are greater than or equal to the frequency domain orthogonality threshold to a preset maximum constant, thereby marking the corresponding AI task combinations as incompatible; the global orthogonal compatibility scheduling module 102 retains the interference cost values ​​in the global interference cost matrix that are less than the frequency domain orthogonality threshold, and uses the interference cost values ​​less than the frequency domain orthogonality threshold as orthogonal compatibility connection edges. By constructing a global interference cost matrix and using the frequency domain orthogonality threshold to perform numerical filtering, task combinations with high-frequency microarchitecture resource contention risks are eliminated, reducing the computational space of subsequent node matching algorithms.

[0072] In step S204, the global orthogonal compatibility scheduling module 102 maps the updated global interference cost matrix to an undirected weighted graph; the global orthogonal compatibility scheduling module 102 maps the AI ​​tasks to be scheduled to vertices in the undirected weighted graph; the global orthogonal compatibility scheduling module 102 maps orthogonal compatible connecting edges to edges in the undirected weighted graph, and maps the retained interference cost values ​​to the weights corresponding to the edges; the global orthogonal compatibility scheduling module 102 obtains the hardware resource capacity constraint information of the physical computing nodes; the global orthogonal compatibility scheduling module 102 uses the hardware resource capacity constraint information as boundary constraints and utilizes a graph partitioning algorithm to partition the undirected... The weighted graph performs minimum weighted graph partitioning calculation to obtain a vertex partitioning set that satisfies the boundary constraints. The global orthogonal compatibility scheduling module 102 distributes multiple artificial intelligence tasks assigned to the same set to the same physical computing node based on the vertex partitioning set. By converting the global interference cost matrix into an undirected weighted graph and introducing hardware resource capacity constraint information, the minimum weighted graph partitioning calculation is performed. This achieves orthogonal compatible combination deployment of multiple artificial intelligence tasks at the macro level of the cluster, effectively alleviating the bus frequency competition conditions within the physical computing node from the source of software scheduling, while ensuring that the allocation result does not exceed the physical carrying capacity limit of the underlying hardware.

[0073] Example 5

[0074] Based on Example 2, this example combines the appendix. Figure 7 , Figure 8 , Figure 9 and Figure 10 The detailed execution process of steps S300 to S500 in Embodiment 2 is explained below. Specifically, the on-chip micro-spectrum monitoring module 103 performs microsecond-level time-domain signal acquisition and spectrum drift and resonance early warning detection during the task operation phase, and the hardware-level dynamic instruction interleaving control module 104 performs a complete hardware-level closed-loop control link based on the detection results, including nanosecond-level instruction pause adaptive injection and dynamic operator fission.

[0075] Please refer to Figure 7 The on-chip micro-spectrum monitoring module 103 performs the following steps for real-time acquisition of microsecond-level time-domain signals based on HPM:

[0076] In step S301, when the AI ​​task is started by the AI ​​acceleration chip on the physical computing node, the on-chip micro-spectrum monitoring module 103 obtains the hardware context identifier corresponding to the AI ​​task and initializes the on-chip hardware performance counter inside the AI ​​acceleration chip according to the hardware context identifier. The on-chip micro-spectrum monitoring module 103 binds the sampling trigger clock source of the on-chip hardware performance counter to the hardware microsecond timer of the AI ​​acceleration chip. In each trigger cycle of the hardware microsecond timer, the on-chip micro-spectrum monitoring module 103 performs event filtering based on the hardware context identifier, directly reads the hardware count value in the last-level cache missing event register and the system bus transaction byte register, obtains the hardware context identifier to bind a specific AI task, binds the sampling trigger clock source to the hardware microsecond timer and directly reads the filtered hardware count value, bypasses the interrupt scheduling mechanism of the host operating system, eliminates the system context switching time consumption caused by software sampling, and realizes high-frequency monitoring of the underlying microarchitecture status at the microsecond level for specific tasks.

[0077] In step S302, the on-chip micro-spectrum monitoring module 103 calculates the last-level cache miss rate and system bus bandwidth usage of the artificial intelligence task in each trigger cycle based on the hardware count value and the trigger cycle time of the hardware microsecond timer. The on-chip micro-spectrum monitoring module 103 then performs normalization processing on the last-level cache miss rate and system bus bandwidth usage according to a preset upper limit threshold for the last-level cache miss rate and the peak bus bandwidth value. Finally, the on-chip micro-spectrum monitoring module 103 performs a weighted sum of the normalized last-level cache miss rate and system bus bandwidth usage within the same trigger cycle to obtain the single-point instantaneous access... The on-chip micro-spectrum monitoring module 103 concatenates the instantaneous memory access load values ​​of single points within multiple consecutive trigger cycles into a discrete dynamic running time series according to the order of trigger time. The on-chip micro-spectrum monitoring module 103 stores the dynamic running time series into the static random access memory inside the artificial intelligence acceleration chip, eliminates dimensional differences by performing normalization processing, and calculates the instantaneous memory access load values ​​of single points to be concatenated into a dynamic running time series. This converts the scattered underlying hardware counting events into continuous time-domain waveform data, providing a format-matched data source for subsequent time-frequency conversion at the chip hardware level.

[0078] Furthermore, after completing the time-domain signal acquisition, the on-chip micro-spectrum monitoring module 103 further performs on-chip hardware frequency domain conversion and spectrum drift and resonance early warning detection. Please refer to [link to relevant documentation]. Figure 8 The on-chip micro-spectrum monitoring module 103 performs the following steps for on-chip hardware frequency domain conversion and spectrum drift and resonance early warning detection:

[0079] In step S303, the on-chip micro-spectrum monitoring module 103 reads the dynamic running-state time series stored in the static random access memory of the physical computing node; the on-chip micro-spectrum monitoring module 103 inputs the dynamic running-state time series into the fast Fourier transform logic circuit embedded inside the artificial intelligence acceleration chip, the fast Fourier transform logic circuit performs hardware-level time-frequency conversion operation, and outputs a frequency domain complex signal; the on-chip micro-spectrum monitoring module 103 uses the on-chip hardware computing unit to perform a square root operation on the real and imaginary parts of the frequency domain complex signal, and extracts the running vibration corresponding to each frequency component. The on-chip micro-spectrum monitoring module 103 outputs a dynamic operating spectrum containing each frequency component and its corresponding operating amplitude. For the hardware implementation structure of the butterfly operation unit and the rotation factor multiplier in the fast Fourier transform logic circuit, those skilled in the art can refer to the existing digital signal processing chip architecture. The corresponding hardware circuit wiring method is a well-known technology in the field and will not be described in detail here. The dynamic operating time series is directly processed and the operating amplitude is extracted by the solidified fast Fourier transform logic circuit, avoiding the instruction pipeline overhead of general processors and meeting the real-time computing requirements of microsecond-level microarchitecture status monitoring.

[0080] Step S304: The on-chip micro-spectrum monitoring module 103 extracts frequency components with amplitudes greater than preset amplitude boundary values ​​from the dynamic operating spectrum, constructing a dominant operating frequency set; the on-chip micro-spectrum monitoring module 103 reads the static spectrum feature metadata from the executable binary file corresponding to the artificial intelligence task; the on-chip micro-spectrum monitoring module 103 obtains the preset standard discrete frequency interval sequence of the physical computing node; the on-chip micro-spectrum monitoring module 103 aligns and maps the frequency components in the dominant operating frequency set and the feature frequency values ​​in the static spectrum feature metadata to the standard discrete frequency interval sequence, generating a dynamic operating spectrum vector and a static feature reference vector; the on-chip micro-spectrum monitoring module 103 performs a cosine similarity calculation operation on the dynamic operating spectrum vector and the static feature reference vector; the on-chip micro-spectrum monitoring module... 103 calculates the dot product of the dynamic running spectrum vector and the static feature reference vector through the on-chip hardware multiply-accumulate unit, and divides the dot product by the product of the magnitude of the dynamic running spectrum vector and the magnitude of the static feature reference vector to obtain the cosine similarity as the spectrum similarity value. The on-chip micro spectrum monitoring module 103 writes the set of dominant running frequencies into the global shared state register array inside the artificial intelligence acceleration chip. When the spectrum similarity value is lower than the preset drift threshold, the on-chip micro spectrum monitoring module 103 determines that the artificial intelligence task is in a spectrum drift state. It generates an alignment vector by mapping discrete frequencies to a standard discrete frequency interval sequence, and calculates the dot product at the hardware level to obtain the spectrum similarity value. This quantifies the deviation of the underlying memory access behavior of the artificial intelligence task and provides a trigger basis for subsequent hardware-level reordering of resources.

[0081] In step S305, if it is determined that there is an artificial intelligence task in a spectrum drift state within the physical computing node, the on-chip micro-spectrum monitoring module 103 reads the global shared state register array and extracts the dominant operating frequency values ​​of other artificial intelligence tasks that are concurrently executing within the same clock cycle within the physical computing node through the global shared state register array; the on-chip micro-spectrum monitoring module 103 performs pairwise registration comparison to obtain the absolute value of the difference between the dominant operating frequency values ​​of any two artificial intelligence tasks; if the absolute value of the difference is less than the preset same-frequency tolerance value, the on-chip micro-spectrum monitoring module 103 determines the corresponding Two artificial intelligence tasks enter a state of resonance; the on-chip micro-spectrum monitoring module 103 acquires the hardware context identifiers of the two artificial intelligence tasks in the state of resonance; the on-chip micro-spectrum monitoring module 103 transmits the resonance warning signal with the hardware context identifier to the hardware-level dynamic instruction interleaving control module 104. By introducing a global shared state register array to cross hardware context isolation, the module compares the dominant operating frequency value and sends the resonance warning signal with the hardware context identifier, enabling the system layer to intercept and control before actual congestion occurs on the physical bus, thus ensuring the execution throughput of computing resources.

[0082] Furthermore, after the on-chip micro-spectrum monitoring module 103 detects a resonance state, the hardware-level dynamic instruction interleaving control module 104 performs nanosecond-level phase difference calculation and instruction pause adaptive injection, referring to... Figure 9 The hardware-level dynamic instruction interleaving control module 104 performs nanosecond-level phase difference calculation and instruction pause adaptive injection, including the following steps:

[0083] In step S401, the hardware-level dynamic instruction interleaving control module 104 receives the resonance warning signal and corresponding hardware context identifier transmitted by the on-chip micro-spectrum monitoring module 103; the hardware-level dynamic instruction interleaving control module 104 obtains the first artificial intelligence task and the second artificial intelligence task that are in the resonance state according to the hardware context identifier; the hardware-level dynamic instruction interleaving control module 104 obtains the most recent memory access bandwidth peak timestamp of the first artificial intelligence task and the second artificial intelligence task; the hardware-level dynamic instruction interleaving control module 104, through the hardware phase detector logic circuit inside the artificial intelligence acceleration chip, combines the dominant operating frequency corresponding to... The physical cycle span is used to map the time difference between the current clock time and the most recent memory access bandwidth peak timestamp to a phase angle of 0 to 2π, thereby extracting the instantaneous physical phase values ​​of the first and second artificial intelligence tasks at the dominant operating frequency; the hardware-level dynamic instruction interleaving control module 104 calculates the absolute value of the difference between the instantaneous physical phase values ​​of the first and second artificial intelligence tasks to obtain the instantaneous physical phase difference. By obtaining the peak timestamp and combining it with the time difference to map the phase angle, nanosecond-level microarchitecture state quantization is achieved, providing definite physical data support for the control of the underlying instruction execution timing;

[0084] The hardware phase detector logic circuit can be composed of a digital comparator, a phase accumulator and a lookup table method arctangent operation unit. Its hardware implementation is a well-known technology in the field and will not be described in detail here.

[0085] In step S402, the hardware-level dynamic instruction interleaving control module 104 compares the instantaneous physical phase difference with a preset safety inversion threshold; if the instantaneous physical phase difference is less than the safety inversion threshold, the hardware-level dynamic instruction interleaving control module 104 determines that the underlying memory access requests of the first artificial intelligence task and the second artificial intelligence task are in a time-overlapping conflict state; the hardware-level dynamic instruction interleaving control module 104 combines the safety inversion threshold, the instantaneous physical phase difference, the dominant operating frequency, and the core clock frequency of the artificial intelligence acceleration chip to calculate the number of pause ticks that need to be injected.

[0086] The formula for calculating the number of pauses is as follows:

[0087]

[0088] In the formula, Indicates the number of pause ticks that need to be injected; This indicates the preset safety inversion threshold; This represents the instantaneous physical phase difference between the first and second artificial intelligence tasks; The dominant operating frequency value indicating that the first and second artificial intelligence tasks are in a state of resonance; This represents the core clock frequency value of the AI ​​acceleration chip; This indicates an up-rounding operation. The above formula is used to calculate the number of pauses, and the overlap difference of the physical phase is strictly converted into the discrete clock cycle compensation value at the chip level to ensure the accuracy and executability of the peak shift adjustment parameters.

[0089] In step S403, the hardware-level dynamic instruction interleaving control module 104 obtains the system priority identifiers of the first artificial intelligence task and the second artificial intelligence task; the hardware-level dynamic instruction interleaving control module 104 sets the artificial intelligence task with the lower system priority identifier as a non-critical task; the hardware-level dynamic instruction interleaving control module 104 injects no-operation instructions equal to the number of pause ticks into the hardware launch queue corresponding to the non-critical task, or the hardware-level dynamic instruction interleaving control module 104 lowers the launch enable level signal of the hardware launch queue corresponding to the non-critical task and maintains a clock cycle equal to the number of pause ticks. By injecting no-operation instructions into the hardware launch queue of the non-critical task or lowering the launch enable level signal, the underlying instruction launch time of the non-critical task is delayed without suspending the operating system process or performing task node migration, forcing the memory access pulses of multiple artificial intelligence tasks to be interleaved on the time axis, effectively alleviating the physical congestion of the system bus at the microarchitecture level.

[0090] Furthermore, when pause injection fails to continuously eliminate bus conflicts, the system further triggers operator fission to reconstruct the physical frequency, referring to... Figure 10 The hardware-level dynamic instruction interleaving control module 104 and the host operating system perform multi-task duty cycle out-of-bounds detection and dynamic operator fission steps, including:

[0091] Step S501: The hardware-level dynamic instruction interleaving control module 104 counts the cumulative number of pauses continuously injected into the hardware launch queue of non-critical tasks within a preset time window; the hardware-level dynamic instruction interleaving control module 104 obtains the hardware tolerance upper limit threshold preset inside the physical computing node; wherein, the memory access pulse duty cycle is defined as the ratio of the actual number of execution clock cycles of the non-critical task within the preset time window to the total number of clock cycles in the window; when the cumulative number of pauses continues to increase, the duty cycle decreases accordingly; when the cumulative number of pauses is greater than or equal to the hardware tolerance upper limit threshold, the hardware-level dynamic instruction interleaving control module 104 determines that the memory access pulse duty cycle between the non-critical task and other artificial intelligence tasks in the same frequency resonance state has exceeded the limit, and by counting the cumulative number of pauses and comparing it with the hardware tolerance upper limit threshold, it quantitatively evaluates the physical effectiveness of the phase interleaving control at the microarchitecture level and identifies the dense bus conflict state that cannot be eliminated by simply relying on hardware launch delay;

[0092] In step S502, if the memory access pulse duty cycle is determined to be out of bounds, the hardware-level dynamic instruction interleaving control module 104 sends a hardware interrupt signal to the host operating system of the physical computing node through the peripheral component interconnect bus; the host operating system receives the hardware interrupt signal and parses the hardware context identifier of the non-critical task carried by the hardware interrupt signal; the host operating system suspends the device flow execution queue corresponding to the non-critical task according to the hardware context identifier; the host operating system intercepts the target tensor operator currently in the execution state of the non-critical task and records the current memory read pointer and calculation progress status information of the target tensor operator. By sending a hardware interrupt signal and suspending the device flow execution queue, the underlying physical instruction issuance operation of the conflicting task is intercepted at the software framework layer, providing a safe intervention control point for subsequent reconstruction of the underlying computing logic without terminating the non-critical task;

[0093] In step S503, the host operating system reads the preset original computation block size of the target tensor operator; based on the current memory read pointer and computation progress status information, the host operating system determines the remaining tensor data region that the target tensor operator has not yet executed; the host operating system re-executes the slicing operation on the remaining tensor data region in the spatial dimension, dividing the original computation block size into multiple microkernel computation blocks with reduced size; the host operating system equivalently decomposes the target tensor operator into multiple sub-operator sequences corresponding to the microkernel computation blocks; the host operating system redeploys the multiple sub-operators to the artificial intelligence acceleration chip of the physical computing node according to the sub-operator sequences to continue execution. By dividing the microkernel computation blocks based on the remaining tensor data region and decomposing the target tensor operator, the temporal distribution of memory access requests of non-critical tasks is changed at the physical level, causing their dominant operating frequency to deviate from the original co-frequency resonance band, fundamentally solving the physical condition of co-frequency resonance between non-critical tasks and other artificial intelligence tasks, and constructing a closed-loop adaptive adjustment mechanism of software and hardware collaboration.

[0094] In this application, "synchronous resonance" refers to the phenomenon that the memory access requests of multiple concurrent artificial intelligence tasks exhibit overlapping dominant frequencies under frequency domain analysis, resulting in the periodic superposition of their memory access pulses in the time domain, which intensifies bus contention.

Claims

1. An AI computing power resource adaptive orchestration system for multi-task scenarios, characterized in that, include: The computation graph semantic parsing and feature injection module (101) is used to receive the artificial intelligence task to be compiled, generate the memory access request sequence corresponding to the artificial intelligence task and convert it into static spectrum feature metadata, and inject the static spectrum feature metadata into the executable binary file corresponding to the artificial intelligence task. The global orthogonal compatibility scheduling module (102) is used to extract static spectral feature metadata from the executable binary files of multiple AI tasks to be scheduled in order to calculate the interference cost, and to distribute multiple AI tasks that meet the orthogonal compatibility conditions to the same physical computing node. The on-chip micro-spectrum monitoring module (103) is used to collect dynamic running time series when the artificial intelligence acceleration chip of the physical computing node performs artificial intelligence tasks, perform hardware-level time-frequency conversion to extract dynamic running spectrum, and send a co-frequency resonance warning signal with hardware context identifier when it is determined that multiple concurrently executed artificial intelligence tasks have entered the same frequency resonance state. The hardware-level dynamic instruction interleaving control module (104) is used to receive the same-frequency resonance early warning signal, extract the instantaneous physical phase difference of the artificial intelligence task in the same-frequency resonance state based on the hardware context identifier, and inject pause cycle instructions into the hardware launch queue corresponding to the non-critical task; when the cumulative number of pause cycle instructions reaches the hardware tolerance upper limit threshold, a hardware interrupt signal is sent to the host operating system to trigger the microkernel operator fission reconstruction of the target tensor operator.

2. The AI ​​computing power resource adaptive orchestration system in a multi-task scenario as described in claim 1, characterized in that, When the computation graph semantic parsing and feature injection module (101) generates the memory access request sequence, it is specifically used for: Extract the directed acyclic graph structure corresponding to the artificial intelligence task, traverse the nodes in the directed acyclic graph structure to identify the tensor operator type, and divide the nodes into computationally intensive operators and memory-intensive operators. The hardware parameter configuration information of the target hardware platform is retrieved based on the distribution sequence of computationally intensive operators and memory-intensive operators, and the microarchitecture simulator is configured based on the hardware parameter configuration information. Tensor operators in the directed acyclic graph structure are input into the microarchitecture simulator according to the topological sorting rules for static deduction, and the ideal memory access request rate of each tensor operator is output. According to the time axis set by topological sorting, the ideal memory access request rates of tensor operators with dependencies are sequentially spliced ​​together, and the ideal memory access request rates of parallel tensor operators without dependencies are superimposed in the time domain to generate a discrete time sequence signal that varies with the processor clock cycle as the memory access request sequence.

3. The AI ​​computing power resource adaptive orchestration system in a multi-task scenario as described in claim 2, characterized in that, When the computation graph semantic parsing and feature injection module (101) generates and injects static spectral feature metadata, it is specifically used for: The length of the memory access request sequence is padded with zeros to a power of 2 and then a discrete Fourier transform operation is performed to convert it into a frequency domain signal. The signal is then sorted in descending order according to the amplitude value. Frequency components whose sorting position is less than or equal to a preset extraction dimension threshold are extracted as feature frequencies and their corresponding amplitude values ​​to construct static spectral feature metadata. The static spectral feature metadata is serialized and encoded using a continuous key-value pair arrangement to generate a binary data stream; An independent custom data storage segment is allocated within the executable binary file to write the binary data stream, and the starting memory offset address and effective data length of the custom data storage segment are recorded in the file header structure information.

4. The AI ​​computing power resource adaptive orchestration system in a multi-task scenario as described in claim 1, characterized in that, When the global orthogonal compatibility scheduling module (102) calculates the interference cost, it is specifically used for: Read the file header structure information of the executable binary file, extract the static spectral feature metadata according to the starting memory offset address and effective data length recorded in the file header structure information, and perform deserialization decoding to obtain the feature frequency value and amplitude value set corresponding to each scheduled artificial intelligence task; By iterating through the combinations, any two AI tasks to be scheduled are selected as the first task and the second task, respectively. The interference cost of the first task and the second task in the frequency domain is calculated based on the following formula (S202): in, Indicates the cost of interference; and These represent the total number of feature frequencies contained in the first task and the second task, respectively. and These represent the amplitude and characteristic frequency values ​​of the first task, respectively. and These represent the amplitude and characteristic frequency values ​​of the second task, respectively. This represents a constant correction factor.

5. The AI ​​computing power resource adaptive orchestration system in a multi-task scenario as described in claim 4, characterized in that, When the global orthogonal compatibility scheduling module (102) executes the deployment, it is specifically used for: A global interference cost matrix is ​​constructed based on the pairwise interference costs between multiple AI tasks to be scheduled. Interference cost values ​​that are greater than or equal to a preset frequency domain orthogonality threshold are updated to a preset maximum constant, and interference cost values ​​that are less than the frequency domain orthogonality threshold are used as orthogonal compatible connection edges. The updated global interference cost matrix is ​​mapped to an undirected weighted graph, where the AI ​​tasks to be scheduled are mapped to vertices, the orthogonal compatible connection edges are mapped to edges, and the corresponding interference cost values ​​are mapped to the edge weights. Obtain the hardware resource capacity constraint information of the physical computing node, and perform minimum weight graph partitioning calculation on the undirected weighted graph using the hardware resource capacity constraint information as boundary constraint conditions to obtain a vertex partition set. Based on the vertex partition set, distribute and deploy multiple artificial intelligence tasks partitioned into the same set to the same physical computing node.

6. The AI ​​computing power resource adaptive orchestration system in a multi-task scenario as described in claim 1, characterized in that, When the on-chip micro-spectrum monitoring module (103) acquires dynamic operating time series data, it is specifically used for: Obtain the hardware context identifier corresponding to the artificial intelligence task, initialize the on-chip hardware performance counter according to the hardware context identifier, and bind its sampling trigger clock source to the hardware microsecond timer; Within each trigger cycle of the hardware microsecond timer, event filtering is performed based on the hardware context identifier, and the hardware count values ​​in the last-level cache miss event register and the system bus transaction byte register are directly read to calculate the last-level cache miss rate and the system bus bandwidth usage. Normalization is performed based on the preset upper limit threshold of the last-level cache miss rate and the peak bandwidth value of the bus. The normalized last-level cache miss rate and the system bus bandwidth usage are weighted and summed to obtain the instantaneous memory access load value at a single point. The instantaneous memory access load values ​​at a single point in multiple consecutive trigger cycles are spliced ​​together in the order of trigger time to form the dynamic running time series.

7. The AI ​​computing power resource adaptive orchestration system in a multi-task scenario as described in claim 6, characterized in that, When the on-chip micro-spectrum monitoring module (103) sends a resonance warning signal, it is specifically used for: The dynamic running time series is input to the fixed fast Fourier transform logic circuit to output a frequency domain complex signal. The on-chip hardware computing unit performs a square root operation on the real part and the imaginary part to extract the running amplitude. The frequency components with running amplitudes greater than the preset amplitude boundary value are extracted to construct the dominant running frequency set. Obtain a standard discrete frequency interval sequence, align and map the frequency components in the dominant operating frequency set and the feature frequency values ​​in the static spectral feature metadata to the standard discrete frequency interval sequence, generate a dynamic operating spectrum vector and a static feature reference vector, and calculate the cosine similarity between the two vectors to obtain the spectrum similarity value. When the spectral similarity value is lower than the preset drift threshold, the dominant operating frequency set is written into the global shared state register array; the dominant operating frequency values ​​of other concurrently executed artificial intelligence tasks are obtained by reading the global shared state register array; if the absolute value of the difference between the dominant operating frequency values ​​of any two artificial intelligence tasks is less than the preset same-frequency tolerance value, it is determined that the same-frequency resonance state has been entered and a warning signal is sent.

8. The AI ​​computing power resource adaptive orchestration system in a multi-task scenario as described in claim 1, characterized in that, The hardware-level dynamic instruction interleaving control module (104) is specifically used for: Obtain the most recent memory access bandwidth peak timestamps of the first and second AI tasks, combine them with the physical cycle span corresponding to the dominant operating frequency, map the time difference between the current clock time and the most recent memory access bandwidth peak timestamp into a phase angle, extract the instantaneous physical phase value, and then calculate the absolute value of the difference to obtain the instantaneous physical phase difference; Based on the safety inversion threshold, instantaneous physical phase difference, dominant operating frequency value, and core clock frequency value of the artificial intelligence acceleration chip, the number of pause ticks is obtained by rounding up using a preset calculation formula. The AI ​​tasks with lower system priority are set as non-critical tasks. No-operation instructions are injected into the hardware transmission queue corresponding to the non-critical task in a number equal to the number of pause ticks. Alternatively, the transmit enable level signal of the hardware transmission queue corresponding to the non-critical task is lowered and maintained for a clock cycle equal to the number of pause ticks.

9. The AI ​​computing power resource adaptive orchestration system in a multi-task scenario as described in claim 8, characterized in that, The system also includes a host operating system; The hardware-level dynamic instruction interleaving control module (104) is also used to count the cumulative number of pauses continuously injected into the hardware transmission queue within a preset time window, and send a hardware interrupt signal to the host operating system when the cumulative number of pauses is greater than or equal to the hardware tolerance upper limit threshold. The host operating system is used to receive the hardware interrupt signal and suspend the device stream execution queue corresponding to the non-critical task, intercept the target tensor operator in the execution state, record the current memory read pointer and calculation progress status information, and determine the remaining tensor data area that has not yet been executed. The host operating system is also used to re-execute the slicing operation on the remaining tensor data region in the spatial dimension to generate multiple microkernel computing blocks with reduced size, and to equivalently decompose the target tensor operator into multiple sub-operator sequences and re-distribute them to the artificial intelligence acceleration chip of the physical computing node for continued execution.

10. The AI ​​computing power resource adaptive orchestration system in a multi-task scenario as described in claim 1, characterized in that, The system also includes: The operator branch scheduling module (105) is used to read the precision channel allocation ratio, divide the input feature tensor into the first feature tensor quantum set and the second feature tensor quantum set along the channel dimension, and allocate high-precision floating-point operator branches and low-precision fixed-point operator branches respectively. The asymmetric precision execution module (106) is used to distribute the calculation instruction sequences of the high-precision floating-point operator branch and the low-precision fixed-point operator branch to the floating-point arithmetic unit and the integer arithmetic unit respectively for processing within the same clock cycle, and write the data branch that is calculated first into the hardware synchronous buffer queue to wait until the end signal is output. The precision alignment and feature fusion module (107) is used to perform dequantization mapping on the low-precision feature tensor result based on the quantization scaling factor and numerical offset after receiving the end signal to obtain the format-aligned feature tensor, and to perform matrix concatenation operation on the high-precision feature tensor result and the format-aligned feature tensor in the channel dimension to obtain the fused feature tensor.