Isomer model mixed reasoning task priority scheduling and computing power optimization method

Abstract

The application discloses a heterogeneous model mixed reasoning task priority scheduling and computing power optimization method. The method is applied to a system comprising a calculation graph analysis module, a state manifold construction module, a control surface solving module and a discretization dispatching module. The method comprises the following steps: S100, receiving a multi-model calculation graph and establishing a discrete tensor operator state; S200, generating a diagonal gating matrix based on a graph theory dependency relationship, mapping and embedding the operator state into a continuous state space model of a heterogeneous resource, and establishing an adaptive state manifold evolution model; the control surface solving module solves a Riccati equation based on the model and a cost functional to output an optimal feedback control law, and analyzes the feedback control law into feedback gain matrix data; the discretization dispatching module converts the gain matrix data into discrete execution instructions and dispatches the discrete execution instructions to a hardware queue. The application breaks through the combination explosion bottleneck, avoids task high-frequency migration oscillation, and realizes the optimal balance of computing power and memory access.

Description

Technical Field

[0001] This invention relates to the fields of computer architecture and artificial intelligence computing technology, specifically to a method for priority scheduling and computing power optimization of heterogeneous model hybrid inference tasks. Background Technology

[0002] With the application of deep learning technology in industrial scenarios, artificial intelligence computing tasks are showing a trend towards more complex model structures and denser concurrent requests. To meet computing demands, computing centers typically employ heterogeneous hardware platforms that include computing units such as central processing units, graphics processing units, and neural network processors. In this heterogeneous hardware architecture, how to allocate resources and schedule timing for concurrent model inference tasks is a crucial aspect of ensuring the utilization rate of the underlying hardware computing power.

[0003] In heterogeneous computing environments, scheduling schemes for hybrid model inference tasks typically rely on discrete heuristic algorithms or static rule-based allocation mechanisms. Upon receiving an inference request, the underlying task scheduler parses the model into a discrete execution sequence composed of multiple tensor operators and dispatches these discrete tensor operators to physical computing nodes based on hardware affinity rules or resource load thresholds. This scheduling mechanism involves combinatorial optimization of the discrete state space; the scheduling system needs to traverse the execution states of operators and the queuing status of hardware resources to determine the flow of tensor tasks.

[0004] However, existing scheduling mechanisms based on discrete state spaces face scheduling difficulties in multi-model concurrent scenarios. When a large number of models run in parallel, the scheduling paths and execution times of discrete tensor operators generate an exponentially increasing number of combined states, leading to a combinatorial explosion problem. This makes discrete scheduling algorithms prone to getting trapped in local optima and increases computational overhead. If the discrete scheduling problem is transformed into a continuous state space problem for mathematical solution, existing continuous modeling methods cannot handle the discrete topological dependencies between tensor operators. Directly introducing discrete topological impedances into the mathematical mapping will cause anomalies in the solution of differential equations, preventing the system from establishing continuous differentiable constraints on the operator execution time at the underlying level. Existing technologies lack a stable foundation for continuous mathematical mapping and cannot introduce optimal control laws, resulting in low scheduling efficiency under concurrent congestion. Summary of the Invention

[0005] To address the problems mentioned in the background, this invention provides a method for priority scheduling and computational power optimization of heterogeneous model hybrid inference tasks. This method is applied to a system for priority scheduling and computational power optimization of heterogeneous model hybrid inference tasks. The system includes a computation graph analysis module, a state manifold construction module, a control surface solution module, and a discretization dispatch module.

[0006] Step S100: The computation graph parsing module receives the multi-model computation graph and establishes the discrete tensor operator states;

[0007] In step S200, the state manifold construction module generates a diagonal gating matrix based on the graph theory dependencies of the tensor operator states, and embeds the tensor operator states into the continuous state space model of heterogeneous hardware resources to establish an adaptive state manifold evolution model. The control surface solution module calculates the solution matrix within a preset period using an algebraic Riccati equation solver based on the adaptive state manifold evolution model and preset computation-memory joint Hamiltonian cost functional parameters, and outputs an adaptive feedback scheduling control law based on the Riccati equation in the continuous time domain. The adaptive feedback scheduling control law is then parsed into feedback gain matrix data. The discretization dispatch module converts the feedback gain matrix data into discrete execution instructions recognizable by the underlying physical hardware and dispatches them to the execution queue of the target heterogeneous hardware node to complete a single tensor-level hardware scheduling.

[0008] Preferably, step S100, the process of establishing the discrete tensor operator state, includes:

[0009] Step S110: The computation graph parsing module receives multiple concurrent deep learning model inference requests, reads the weight file and network structure description file corresponding to the deep learning model, maps each network layer in the network structure description file to the underlying tensor operator node, maps the data transmission path between network layers to directed edges, assigns an independent model bias index to each deep learning model, and merges the graph structures of different models into a global computation graph set based on the model bias index.

[0010] Step S120: The operator feature extraction unit traverses the tensor operator nodes in the global computation graph set, extracts the static physical properties of the tensor operators, obtains the input tensor dimension and the output tensor dimension, and reads the data precision format identifier of the tensor operator node.

[0011] Step S130: The operator feature extraction unit calculates the theoretical computational load and memory access bandwidth overhead required for the tensor operator node to be executed on the heterogeneous hardware node based on the static physical properties, and encapsulates the total theoretical number of floating-point operations and the number of bytes accessed for memory reading and writing into an operator feature vector.

[0012] In step S140, the topology dependency analysis unit establishes the pre-dependencies and post-dependencies of the tensor operator nodes based on the directed edges in the global computation graph set, and generates an adjacency matrix.

[0013] Preferably, step S200, the process of establishing an adaptive state manifold evolution model, includes:

[0014] Step S210: The underlying physical state acquisition unit acquires the underlying physical performance load data of the heterogeneous hardware platform, obtains the number of threads in the waiting execution state, calculates the number of kernel instructions in the queue, and the real-time data transmission throughput.

[0015] Step S220: The continuous equation mapping unit allocates a continuous-time state vector in the state vector register space and writes the underlying physical performance load data into the memory address of the numerical component corresponding to the continuous-time state vector in sequence.

[0016] Step S230: The continuous equation mapping unit constructs a state decay matrix, which maps the physical processing capability evolution benchmark of the heterogeneous hardware platform itself.

[0017] Step S240: The continuous equation mapping unit constructs a control input matrix, and the control input matrix maps the gain transfer path of continuous scheduling instructions to the target physical execution queue;

[0018] In step S250, the continuous equation mapping unit constructs an input perturbation vector with the same dimension as the total number of global tensor operators based on the operator feature vector. The input perturbation vector quantizes and records the hardware resource computation requests attached to the tensor operator nodes at the execution front in the multi-model hybrid inference task.

[0019] Preferably, step S200, the process of generating the diagonal gating matrix, includes:

[0020] Step S2021: The continuous equation mapping unit receives the adjacency matrix, determines all the preceding tensor operators of the target tensor operator, and calculates the completion ratio of the preceding dependencies of the target tensor operator by reading the completion callback flag of the preceding tensor operator in the underlying hardware execution queue.

[0021] Step S2022: The continuous equation mapping unit constructs a composite soft constraint function based on the inverse of the logarithmic barrier function based on the prior dependency completion ratio, and calculates the diagonal elements in the diagonal gating matrix corresponding to each of the target tensor operators. The values ​​of the diagonal elements are determined by the following formula:

[0022]

[0023] In the formula, Characterizing the first in the diagonal gating matrix Line and number The diagonal element at the column intersection; Characterizing the first The percentage of prerequisite dependencies completeness for each tensor operator; Characterizes the obstacle penalty coefficient preset by the hardware system; Characterized by a tolerance constant for the smallest positive real number;

[0024] In step S2023, the continuous equation mapping unit sequentially writes the calculated diagonal elements into the main diagonal memory address of the diagonal gating matrix, and uniformly assigns the value zero to the internal elements at non-main diagonal positions.

[0025] In step S2024, the continuous equation mapping unit performs matrix multiplication on the diagonal gating matrix and the input perturbation vector.

[0026] Preferably, the preset process for the computation-memory joint Hamiltonian cost functional parameters includes:

[0027] Step S301: The control surface solving module initializes the memory space of the cost functional, and the cost functional is configured as a quadratic integral structure containing a quadratic term of the system state vector and a quadratic term of the continuous control input vector.

[0028] In step S302, the control plane solving module constructs a state penalty matrix and configures the state penalty matrix as a positive semi-definite matrix. The state penalty matrix is ​​used to quantify the latency penalty caused by bandwidth contention and computing queue backlog. When constructing the state penalty matrix, the user-preset priority weights carried by the multi-model inference task are extracted, and the priority weights are used to perform weighted operations on the main diagonal elements where the physical state components of the corresponding task are located, so that high-priority tasks have a larger latency penalty coefficient.

[0029] Step S303: The control surface solving module constructs a control penalty matrix and configures the control penalty matrix as a positive definite matrix. The control penalty matrix is ​​used to quantify the communication latency and context switching penalty caused by task switching between heterogeneous hardware nodes.

[0030] Step S304: The control surface solving module, based on the system state vector, the state penalty matrix, the continuous control input vector, and the control penalty matrix, performs infinite time-domain integration on the quadratic form in the continuous time domain to generate a computation-memory joint Hamiltonian cost functional. The mathematical expression of the computation-memory joint Hamiltonian cost functional is as follows:

[0031]

[0032] In the formula, Represents the total cost as a real number; Characterizes the system state vector; The transpose of the system state vector; The state penalty matrix represents the positive semidefinite state. Characterizes the continuous control input vector; The transpose of the continuous control input vector; The control penalty matrix represents positive definiteness; Characterizing time integral variables;

[0033] In step S305, the control surface solving module locks and stores the parameters of the computation-memory joint Hamiltonian cost functional in a high-speed static random access memory.

[0034] Preferably, the heterogeneous model hybrid inference task priority scheduling and computing power optimization system further includes a physically isolated architecture, and the construction process includes:

[0035] Step S401: Call the process affinity interface of the operating system to constrain the control plane solving module to the independent physical processor core corresponding to the shadow control plane, and exclusively bind the discretization dispatch module to the physical processor core corresponding to the high-speed data plane.

[0036] Step S402: Establish a cross-plane global shared memory mapping between the shadow control plane and the high-speed data plane;

[0037] In step S403, the control surface solving module calls a hardware-level atomic write instruction to directly push the feedback gain matrix data into a pre-initialized lock-free circular buffer in the cross-surface global shared memory and executes a memory barrier instruction. The discretization dispatch module polls the write end index pointer of the lock-free circular buffer through a physical memory pointer offset instruction and reads the latest version of the feedback gain matrix data.

[0038] Preferably, the process by which the control surface solver module outputs the adaptive feedback scheduling control law in the continuous time domain includes:

[0039] Step S501: The algebraic Riccati equation solver extracts the continuous equation mapping parameters and the computation-memory joint Hamiltonian cost functional parameters to construct a continuous-time algebraic Riccati equation. The mathematical expression of the continuous-time algebraic Riccati equation is as follows:

[0040]

[0041] In the formula, Characterizing the state decay matrix; The transpose of the state decay matrix; The matrix representing the unique positive definite symmetric solution to be solved; Characterizes the control input matrix; The transpose of the control input matrix; The inverse matrix representing the control penalty matrix; The state penalty matrix represents the state.

[0042] Step S502: The control surface solving module constructs the Hamiltonian matrix and performs real Schul decomposition operation to extract the eigenvector subspace, performs matrix multiplication operation between the lower half matrix component and the inverse matrix of the upper half matrix component, and calculates the solution matrix.

[0043] Step S503: The control surface solving module performs a series multiplication operation on the inverse matrix of the control penalty matrix, the transpose matrix of the control input matrix, and the solution matrix to calculate the feedback gain matrix data, and uses the feedback gain matrix data to generate an adaptive feedback scheduling control law based on the Riccati equation in the continuous time domain.

[0044] In step S504, the control surface solving module converts the feedback gain matrix data into a low-level fixed-point numerical format compatible with the discretization dispatch module.

[0045] Preferably, the process by which the discretization dispatch module converts the instructions into discrete execution instructions recognizable by the underlying physical hardware includes:

[0046] Step S601: The discretization dispatch module receives the feedback gain matrix data transmitted by the lock-free ring buffer, performs matrix multiplication operation on the negative value of the feedback gain matrix data and the continuous time state vector to calculate the continuous dispatch rate value corresponding to each heterogeneous hardware node, and performs non-negative saturation truncation processing on the continuous dispatch rate value to ensure that the rate value input to the integral accumulator logic unit is always non-negative.

[0047] Step S602: The discretization dispatch module configures an independent integral accumulator logic unit for each heterogeneous hardware node. In each hardware clock interrupt cycle, the integral accumulator logic unit multiplies the received continuous dispatch rate value by the step length of the hardware clock interrupt cycle to obtain the single-step integral increment value, and then performs an addition operation with the current accumulated value and writes it back into the register to complete the numerical integral accumulation operation.

[0048] In step S603, the discretization dispatch module determines the amount of hardware computation consumed by the tensor operator to be processed, and establishes a dimensionally unified comparison benchmark with the continuously accumulated value in the logic unit of the integrator accumulator.

[0049] Preferably, the discretization dispatch module includes a hysteresis state barrier determination unit, and the determination process includes:

[0050] Step S701: The hysteresis barrier determination unit reads the cross-node communication delay parameters and memory context switching time values ​​of the heterogeneous hardware platform and performs addition operations to generate the hysteresis barrier constant.

[0051] In step S702, the hysteresis barrier determination unit extracts the hardware computation workload value and compares the source hardware node identifier with the target heterogeneous hardware node identifier. When the source hardware node identifier and the target heterogeneous hardware node identifier are consistent, the hardware computation workload value is used as the barrier determination threshold. When the source hardware node identifier and the target heterogeneous hardware node identifier are inconsistent, the hardware computation workload value is numerically superimposed with the hysteresis barrier constant to generate the barrier determination threshold.

[0052] Preferably, the process of the discretization dispatch module dispatching to the execution queue of the target heterogeneous hardware node includes:

[0053] Step S703: The discretization dispatch module compares the current accumulated value in the integral accumulator logic unit with the barrier determination threshold. When the current accumulated value is strictly less than the barrier determination threshold, the cross-device tensor task migration call instruction is rejected.

[0054] Step S704: When the current accumulated value is greater than or equal to the barrier determination threshold, the discretization dispatch module encapsulates the tensor operator to be processed into an indivisible execution handle and pushes it to the direct memory access command queue of the target heterogeneous hardware node.

[0055] In step S705, after receiving the synchronization response signal from the underlying bus, the discretization dispatch module subtracts the hardware calculation workload value from the current accumulated value to generate the updated integral remaining amount, and writes it back to the execution state reset update in the integral accumulator logic unit.

[0056] The present invention, by adopting the above technical solution, can bring the following beneficial effects:

[0057] 1. This invention embeds discrete tensor operator state mappings into a continuous state-space model of heterogeneous hardware resources and establishes an adaptive state manifold evolution model by combining a logarithmic barrier function diagonal gating matrix generated based on operator dependencies. This overcomes the bottleneck of discrete combinatorial explosion faced by multi-task concurrency and realizes continuous differentiability constraints on operator execution timing at the underlying numerical mapping level. It effectively avoids the equation solution collapse anomaly caused by the introduction of discrete topological impedance and provides a stable mathematical foundation for the introduction of nonlinear optimal control laws.

[0058] 2. This invention achieves physical isolation of computational resources between control law calculation and underlying task dispatch by asynchronously solving the algebraic Riccati equation corresponding to the computation-memory joint Hamiltonian cost functional within a physically isolated architecture, generating feedback gain matrix data, and transferring it to the high-speed data plane using cross-plane shared memory. Without occupying the computing power of the data plane interruption processing, it provides the system with the optimal physical path guidance to avoid communication bus blockage and hardware computing power idleness, ensuring a dynamic optimal balance between underlying bandwidth contention penalty and context switching overhead.

[0059] 3. This invention enables the transformation of continuous optimal control instructions into the execution state of the underlying indivisible discrete operators by configuring an integral accumulator logic unit that executes the continuous dispatch rate integral for heterogeneous hardware nodes and introducing a hysteresis barrier judgment threshold constructed in combination with communication delay overhead. At the same time, it intercepts the high-frequency migration behavior of tensor operators between different computing queues by relying on state barrier condition comparison, effectively suppressing the task jitter oscillation phenomenon caused by cross-device scheduling, and ensuring the continuity of multi-hardware collaborative processing and the stability of data flow from the physical dispatch level. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of the heterogeneous model hybrid inference task scheduling system architecture of the present invention;

[0061] Figure 2 This is a flowchart illustrating the heterogeneous model hybrid inference task priority scheduling and computing power optimization method of the present invention;

[0062] Figure 3 This is a schematic diagram of the feature extraction process of the multi-model computation graph tensor operator in this invention;

[0063] Figure 4 This is a schematic diagram of the heterogeneous resource state space continuous mapping process of the present invention;

[0064] Figure 5 This is a schematic diagram of the process for generating the topological dependency soft constraint matrix based on the logarithmic barrier function according to the present invention;

[0065] Figure 6 This is a schematic diagram of the construction process of the computation-store joint optimal control cost functional of the present invention;

[0066] Figure 7 This is a schematic diagram of the physical isolation architecture between the control plane and the data plane of the present invention;

[0067] Figure 8 This is a schematic diagram of the process for solving the algebraic Riccati equation and generating the optimal control law in this invention;

[0068] Figure 9 This is a schematic diagram of the discretized integral accumulator design for continuous control flow according to the present invention.

[0069] Figure 10 This is a schematic diagram of the process of determining the hysteresis state barrier and the underlying hardware discrete dispatch mechanism of the present invention. Detailed Implementation

[0070] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] Please refer to the appendix. Figure 1 This embodiment provides a method for priority scheduling and computing power optimization of heterogeneous model hybrid inference tasks. The method is implemented based on a system for priority scheduling and computing power optimization of heterogeneous model hybrid inference tasks. The system is deployed on a heterogeneous hardware platform containing a non-uniform memory access architecture. The heterogeneous hardware platform contains physical execution units with multiple instruction set architectures and computing characteristics.

[0072] Specifically, the heterogeneous hardware resource nodes in this embodiment include a central processing unit, a graphics processing unit, and a specific neural network processor. At the memory physical level, the heterogeneous hardware platform has host static memory and device dynamic video memory. Different hardware resource nodes communicate with each other through a high-speed peripheral component interconnect bus.

[0073] The heterogeneous model hybrid inference task priority scheduling and computing power optimization system mainly includes a computation graph parsing module 10, a state manifold construction module 20, a control plane solving module 30, and a discretization dispatch module 40. The above functional modules are physically implemented by relying on the host machine's operating system kernel process, hardware driver layer, and coprocessing unit to build together.

[0074] The computation graph parsing module 10 is configured to receive multiple deep learning model inference tasks with concurrent input and parse the multiple deep learning model inference tasks into a directed acyclic graph. The computation graph parsing module 10 includes an operator feature extraction unit 11 and a topology dependency analysis unit 12. The operator feature extraction unit 11 is used to parse the node entities in the directed acyclic graph and extract the basic physical properties of each tensor operator. Here, it is explained that the basic physical properties of the tensor operator specifically include the input and output tensor dimensions, floating-point numerical precision, theoretical total number of floating-point operations, and number of bytes accessed for memory reading and writing. The topology dependency analysis unit 12 reads the set of directed edges between nodes in the directed acyclic graph and generates the pre- and post-dependent adjacency matrices of each tensor operator in the computation time sequence.

[0075] The state manifold construction module 20 is communicatively connected to the computation graph analysis module 10 and configured to construct a continuous-time state-space model. The state manifold construction module 20 includes a low-level physical state acquisition unit 21 and a continuous equation mapping unit 22. The low-level physical state acquisition unit 21 reads the performance monitoring hardware registers of the heterogeneous hardware platform to obtain the task queue backlog length and peripheral bus bandwidth occupancy status of each heterogeneous resource node in real time. The continuous equation mapping unit 22 establishes an adaptive state manifold evolution model with the introduction of a logarithmic barrier function based on the acquired low-level physical state and the topological dependency adjacency matrix of the tensor operator mentioned above. In order to obtain the entity feature support of the above mathematical model, the heterogeneous hardware platform is configured with a state vector register space in the low-level storage. The state vector register space stores the system state vector. The physical meaning components of the system state vector are specifically mapped to: the queue instruction length of the CPU computing thread queue, the number of computing cores suspended by the GPU stream multiprocessor, and the real-time data transmission rate of the high-speed communication bus port. The logarithmic barrier function acts on the input update side of the system state vector in the continuous-time state-space model in the form of a diagonal gating matrix. Based on the adjacency matrix generated by the topology dependency analysis unit 12, when a preceding tensor operator fails to return a callback signal indicating completion, the diagonal gating matrix sets the state input gain of the corresponding following tensor operator's dimension to a positive real number close to zero. At the physical implementation level, when the state input gain is lower than a preset threshold, the scheduling system uses the operating system's process blocking mechanism to physically block the path of the following tensor operator into the hardware execution queue.

[0076] The control plane solving module 30 is deployed in a pre-defined physical resource isolation zone within a heterogeneous hardware platform. The physical resource isolation zone is specifically implemented as an independent central processing unit core bound to the operating system. The control plane solving module 30 is configured to asynchronously execute the mathematical solution process of the optimal control law in the background of the operating system. The storage space of the control plane solving module 30 is pre-configured with a computation-memory joint Hamiltonian cost functional model. The computation-memory joint Hamiltonian cost functional model contains semi-positive definite feature matrix data reflecting bandwidth contention penalty and positive definite feature matrix data reflecting hardware computing power context switching power consumption penalty. The control plane solving module 30 integrates an algebraic Riccati equation solver. The algebraic Riccati equation solver reads the adaptive state manifold evolution model and the computation-memory joint Hamiltonian cost functional parameters, and calculates the solution matrix within a preset period using a matrix decomposition algorithm. Based on this, the control plane solving module 30 outputs the adaptive feedback scheduling control law in the continuous time domain and parses the adaptive feedback scheduling control law in the continuous time domain into feedback gain matrix data, which is periodically written into a global shared memory accessible to all hardware nodes.

[0077] The discretization dispatch module 40 is deployed in the data throughput link of the scheduling system and establishes a high-speed read channel with the global shared memory. The discretization dispatch module 40 is configured to convert the continuous scheduling control law output by the control plane solver module 30 into discrete execution instructions recognizable by the underlying physical hardware. The discretization dispatch module 40 internally includes an integral accumulator logic unit 41, which performs time-dimensional numerical integration on the continuous dispatch rate value received by the target heterogeneous node based on the hardware clock interrupt signal of the operating system. The discretization dispatch module 40 also includes a hysteresis state barrier determination unit 42. 2. Read the total theoretical floating-point operations of the tensor operator to be processed, use the total theoretical floating-point operations of the tensor operator to be processed as the hardware computation workload and add the system tolerance constant to generate the barrier judgment threshold. When the current register value of the integrator accumulator logic unit 41 is greater than or equal to the barrier judgment threshold, the discretization dispatch module 40 triggers the application programming interface call instruction of the underlying hardware driver, dispatches the physical memory pointer of the tensor operator to be processed and the calculation kernel function to the execution queue of the target heterogeneous hardware node in the form of indivisible instructions, and simultaneously triggers the clear reset signal of the integrator accumulator logic unit 41 to complete a single tensor-level hardware scheduling.

[0078] Overall Method Process

[0079] Please refer to the appendix. Figure 2 The following, in conjunction with the accompanying drawings, provides a detailed description of the method execution flow, computation graph analysis details, and continuous mapping mechanism provided in this embodiment of the invention. After receiving the inference call instruction from the underlying hardware, the heterogeneous model hybrid inference task priority scheduling and computing power optimization system triggers the execution of subsequent method steps:

[0080] Step S100: The computation graph parsing module 10 receives the multi-model computation graph and establishes the discrete tensor operator states.

[0081] Specifically, step S100 further includes the following specific steps at the underlying driver logic and data processing level:

[0082] Step S101: The computation graph parsing module 10 receives multiple deep learning model inference tasks concurrently on a heterogeneous hardware platform. The underlying kernel of the system assigns an independent process identifier to each deep learning model. The computation graph parsing module 10 calls the internal operator feature extraction unit 11 to load the model inference task carrying the independent process identifier into the host static memory and parse the model inference task carrying the independent process identifier into the corresponding directed acyclic graph set.

[0083] In step S102, the operator feature extraction unit 11 traverses each tensor operator node in the directed acyclic graph set. The operator feature extraction unit 11 reads the physical attribute pointers pre-allocated in the underlying video memory of each tensor operator node, extracts the dimension parameters and data precision format of the tensor operator, and calculates the total number of theoretical floating-point operations required for the tensor operator to be executed on the target hardware, as well as the number of memory access read / write bytes required for memory loading and result writing back, based on the extracted dimension parameters and data precision format. The total number of theoretical floating-point operations required for the tensor operator to be executed on the target hardware and the number of memory access read / write bytes are packaged and formatted into a preset standard data structure and stored in the cache as the numerical basis for subsequent continuous mapping.

[0084] In step S103, the topology dependency analysis unit 12 inside the computation graph parsing module 10 reads the directed edge information in the directed acyclic graph and analyzes the data flow direction between each tensor operator. Based on the data flow direction obtained from the parsing, the topology dependency analysis unit 12 generates an adjacency matrix that represents the order of execution of the operators before and after execution. The adjacency matrix is ​​used to constrain the execution timing of the reasoning task at the logical level.

[0085] In step S200, the state manifold construction module 20 generates a diagonal gating matrix based on the graph theory dependencies of each tensor operator, and embeds the discrete tensor operator state mapping into the continuous state space model of heterogeneous hardware resources.

[0086] Specifically, in step S200, the process of real-time acquisition of the underlying physical state and mapping of the continuous equation includes the following steps:

[0087] In step S201, the underlying physical state acquisition unit 21 in the state manifold construction module 20 polls the performance monitoring register of the heterogeneous hardware platform through the operating system call interface. The underlying physical state acquisition unit 21 obtains the current computing power load queuing data of each heterogeneous resource node such as the central processing unit, graphics processing unit and neural network processor, as well as the throughput occupancy data of the peripheral component interconnection bus. The continuous equation mapping unit 22 allocates the continuous time state vector in the state vector register space to complete the homomorphic mapping of the discrete heterogeneous hardware resource state to the continuous time state vector.

[0088] In step S202, the continuous equation mapping unit 22 constructs an adaptive state manifold evolution model that incorporates a logarithmic barrier function. The dynamic evolution of the system's underlying hardware state is transformed into a differential equation relationship. In order to mathematically bind the scheduling operation with the consumption of underlying hardware resources and to combine the adjacency matrix to realize topological impedance mapping, the continuous equation mapping unit 22 configures and calculates the following state equations in memory:

[0089]

[0090] In the above equation of state, the mapping relationship between each physical quantity and its mathematical symbol is defined as follows: It represents the continuous-time state vector, specifically quantifying the total number of backlogged instructions in the task queues of each heterogeneous hardware node and the bandwidth utilization of the communication bus. Characterizing the continuous-time state vector in relation to time variables The first derivative reflects the real-time rate of change of the physical state of the hardware resources; The state decay matrix characterizes the system. The constant elements of the state decay matrix are set to the natural state decay rate of the heterogeneous hardware node when there is no external task dispatch, and are negative. The natural state decay rate is determined by the ratio of the theoretical floating-point throughput of the computing resource node to the baseline value of the task queue capacity of the computing resource node, and the dimension is time to the negative first power. The control input matrix is ​​used to quantify the resource mapping gain relationship between system control instructions and the execution queues of each physical hardware. The continuous control input vector represents the real-time continuous rate at which the discretization dispatch module 40 distributes tensor computation tasks to each heterogeneous node. The input perturbation vector is represented by a quantized record of the computational load request rate component and the memory access load request rate component that flow into the system cache pool from the parallel branches at the execution front in the multi-model hybrid inference task.

[0091] It should be noted that the posterior dimension component of the system state vector x(t) includes an operator execution state component, which records the execution completion callback flags of each preceding tensor operator; the diagonal gating matrix The completion ratio of the preceding dependencies is calculated based on the operator execution state components. It is obtained by weighted summation and normalization of the callback identifiers of the corresponding preceding tensor operators in the operator execution state components;

[0092] The representation operator-resource topology mapping matrix has the following dimensions: ,in The total number of tensor operators is represented by the operator-resource topology mapping matrix, which is used to map the input perturbation vector in the operator space dimension to the heterogeneous hardware resource dimension where the continuous-time state vector is located through linear projection.

[0093] The diagonal gating matrix, constructed based on the logarithmic barrier function, is characterized by dynamically updated diagonal elements based on the completion status of the preceding operators in the adjacency matrix. When a preceding tensor operator fails to return a callback signal indicating completion, the diagonal gating matrix truncates the path of the corresponding subsequent tensor operator in the input disturbance vector into the continuous-time state vector. Through the underlying numerical mapping mechanism executed by the continuous equation mapping unit 22, the directed acyclic graph node combinatorial scheduling problem is transformed into a continuous state-space control problem. The memory usage, number of operations, and queue queuing characteristics of the discretized tensor operators are all encapsulated in the continuous-time state vector and the state decay matrix. The implementation of step S202 provides a numerical model basis for the nonlinear optimal control law introduced by the control surface solution module 30, thus avoiding the combinatorial explosion defect faced by traditional discrete heuristic scheduling algorithms in multi-task concurrency from the underlying data architecture level.

[0094] Example 1: Feature Extraction and Topological Dependency Analysis of Tensor Operators in Multi-Model Computational Graphs

[0095] Please refer to the appendix. Figure 3 This document provides a detailed description of the specific implementation method for tensor operator feature extraction performed by the computation graph parsing module 10. It should be noted that this embodiment is a further refinement of steps S100 to S103; the technical essence of both is the same, differing only in the data reading path and attribute extraction granularity. Specifically, it includes the following steps.

[0096] In step S110, the computation graph parsing module 10 allocates an independent memory workspace in the user-mode process of the host machine, receives multiple inference requests from deep learning models issued concurrently, reads the weight file and network structure description file corresponding to the deep learning model, maps each network layer in the network structure description file to the underlying tensor operator node, and maps the data transmission path between network layers to directed edges. In the multi-model concurrent scenario, the computation graph parsing module 10 allocates an independent model bias index for each deep learning model, and merges the graph structures of different models into the global computation graph set according to the model bias index.

[0097] In step S120, the operator feature extraction unit 11 traverses the tensor operator nodes in the global computation graph set and extracts the static physical properties of the tensor operators. The operator feature extraction unit 11 accesses the underlying memory allocation table and reads the input tensor pointer and output tensor pointer bound to the tensor operator. Based on the input tensor pointer and output tensor pointer, the operator feature extraction unit 11 obtains the input tensor dimension and output tensor dimension. The specific lower-level features of the input tensor dimension include batch size, number of channels, feature map height, and feature map width. The operator feature extraction unit 11 reads the data precision format identifier of the tensor operator. The data precision format identifier indicates the data type used by the tensor operator. The specific lower-level features of the data precision format identifier include single-precision floating-point numbers, half-precision floating-point numbers, and eight-bit integers.

[0098] In step S130, the operator feature extraction unit 11 calculates the theoretical computational load and memory access bandwidth overhead required for the tensor operator to execute on heterogeneous hardware nodes based on static physical properties. The operator feature extraction unit 11 calculates the theoretical total number of floating-point operations based on the input tensor dimension, output tensor dimension, and the internal multiplication and addition rules of the operator type. The operator feature extraction unit 11 multiplies the values ​​of each dimension in the input tensor dimension and the output tensor dimension together to obtain the total number of scalar elements. The operator feature extraction unit 11 determines the basic byte length occupied by a single scalar element according to the data precision format identifier. Specifically, when the data precision format identifier is a single-precision floating-point number, the operator feature extraction unit 11 will determine the basic byte length occupied by a single scalar element. The byte length is determined to be four bytes; when the data precision format identifier is a half-precision floating-point number, the operator feature extraction unit 11 determines the basic byte length to be two bytes; when the data precision format identifier is an eight-bit integer, the operator feature extraction unit 11 determines the basic byte length to be one byte. The operator feature extraction unit 11 multiplies the total number of scalar elements by the basic byte length to obtain the number of memory access read / write bytes required for the tensor operator to perform memory loading and result writing back. The theoretical total number of floating-point operations characterizes the computational intensity of the tensor operator, and the number of memory access read / write bytes characterizes the memory access intensity of the tensor operator. The operator feature extraction unit 11 encapsulates the theoretical total number of floating-point operations and the number of memory access read / write bytes into an operator feature vector.

[0099] In step S140, the topology dependency analysis unit 12 establishes the pre-dependencies and post-dependencies of tensor operator nodes based on the directed edges in the global computation graph set. The topology dependency analysis unit 12 initializes a matrix with zero values ​​in memory as an adjacency matrix. The row and column numbers of the two-dimensional matrix correspond to the operator sequence numbers generated by the topology sorting algorithm in the global computation graph set, respectively. The topology dependency analysis unit 12 traverses the directed edges in the global computation graph set. When there is a directed edge from the source tensor operator to the target tensor operator, the topology dependency analysis unit 12 modifies the value of the intersection point of the row corresponding to the source tensor operator and the column corresponding to the target tensor operator in the adjacency matrix to one. The adjacency matrix records the data flow sequence between tensor operators and serves as the data basis for the state manifold construction module 20 to generate the diagonal gating matrix.

[0100] Example 2: Continuous mapping of state space for heterogeneous resources

[0101] Please refer to the appendix. Figure 4 This embodiment provides a detailed explanation of the specific implementation method for the linearization of the heterogeneous resource state space by the state manifold construction module 20. This embodiment further refines steps S200 to S202, used to homomorphically map the underlying discrete hardware state to the basic linearized mathematical entity of the continuous-time state space model, specifically including the following steps:

[0102] In step S210, the underlying physical state acquisition unit 21 collects the underlying physical performance load data of the heterogeneous hardware platform through the hardware abstraction layer call interface provided by the operating system. The underlying physical state acquisition unit 21 reads the task scheduling queue length hardware register of the central processing unit to obtain the number of threads in the waiting execution state. The underlying physical state acquisition unit 21 reads the streaming multiprocessor status monitoring register of the graphics processor to obtain the number of computing kernel instructions in the suspended state. The underlying physical state acquisition unit 21 accesses the direct memory access controller of the peripheral component interconnect bus to obtain the real-time data transmission throughput of the communication bus port.

[0103] Step S220: The continuous equation mapping unit 22 allocates a continuous-time state vector in the state vector register space of the heterogeneous model hybrid inference task priority scheduling and computing power optimization system. The continuous-time state vector is composed of multiple physical state components combined in a fixed-dimensional order. The heterogeneous hardware platform includes... Each computing resource node and The total dimension of the continuous-time state vector across the communication bus links is configured by the heterogeneous model hybrid inference task priority scheduling and computing power optimization system as follows: Dimension, the first dimension in the continuous-time state vector The dimensional computational resource component maps to the task queue backlog length of a single computational resource node, and the latter part of the continuous-time state vector. The bandwidth occupancy status of a single communication bus link is mapped to the communication resource component. The continuous equation mapping unit 22 sequentially writes the collected thread count, kernel instruction queue count, and real-time data transmission throughput into the memory address of the corresponding numerical component of the continuous time state vector.

[0104] Step S230: The continuous equation mapping unit 22 constructs a state decay matrix. The state decay matrix maps the physical processing capability evolution benchmark of the heterogeneous hardware platform itself. The dimension of the continuous equation mapping unit 22 is initialized as follows: The diagonal matrix, for the first The diagonal elements corresponding to the dimensional computing resource components are set by the continuous equation mapping unit 22 to the state natural decay rate of the corresponding computing resource node, and are set to negative values. The state natural decay rate is determined by the ratio of the theoretical floating-point throughput of the computing resource node to the baseline value of the task queue capacity of the computing resource node, and its dimension is the negative first power of time. For the subsequent... The diagonal elements corresponding to the dimensional communication resource components are set by the continuous equation mapping unit 22 to the natural decay rate of the corresponding communication bus link, and are negative. The natural decay rate is determined by the ratio of the maximum data transmission rate per unit time of the communication bus link to the bandwidth capacity baseline value of the communication bus link, and its dimension is time to the negative first power. The above negative real number represents the physical process of backlog queue decay caused by the inherent computing power processing execution and the natural release of the memory bus bandwidth of the heterogeneous hardware platform when there is no external task input intervention.

[0105] In step S240, the continuous equation mapping unit 22 constructs a control input matrix. The control input matrix maps the gain transfer path of the continuous scheduling instructions issued by the discretization dispatch module 40 to the target physical execution queue. The number of rows in the control input matrix is ​​equal to the dimension value of the continuous time state vector, and the number of columns in the control input matrix is ​​equal to the total number of hardware computing nodes that the heterogeneous hardware platform supports for independent scheduling. The continuous equation mapping unit 22 configures the internal elements of the control input matrix according to the topology connection relationship of the hardware physical motherboard. When the continuous scheduling instruction channel points to the target heterogeneous hardware node, the element at the intersection of the row corresponding to the target heterogeneous hardware node and the corresponding scheduling communication link in the control input matrix is ​​assigned the value of the hardware preset resource conversion mapping coefficient. The remaining elements in the control input matrix without physical connection paths are uniformly assigned the value of zero.

[0106] In step S250, the continuous equation mapping unit 22 constructs an input perturbation vector. This input perturbation vector quantizes and records the hardware resource computation requests associated with the tensor operators at the execution front in the multi-model hybrid inference task. The continuous equation mapping unit 22 reads the operator feature vector generated by the operator feature extraction unit 11. The continuous equation mapping unit 22 extracts the theoretical total number of floating-point operations and the number of bytes accessed for memory read / write. The continuous equation mapping unit 22 reads the theoretical maximum floating-point throughput constant and the maximum memory physical bandwidth constant pre-configured on the heterogeneous hardware platform. The continuous equation mapping unit 22 divides the theoretical total number of floating-point operations by the theoretical maximum floating-point throughput constant. The normalized time reference value of the computing load is obtained; the continuous equation mapping unit 22 multiplies the normalized time reference value of the computing load by the operator request arrival frequency coefficient per unit time to generate the computing load request rate component; similarly, the continuous equation mapping unit 22 generates the memory access load request rate component based on the number of bytes accessed for memory read and write and the maximum memory physical bandwidth constant; the continuous equation mapping unit 22 assigns the computing load request rate component and the memory access load request rate component to the corresponding computing load component dimension and memory access load component dimension in the input perturbation vector, and the dimension of the input perturbation vector is consistent with the dimension of the time rate of change of the system state vector. The above continuous time state vector, state decay matrix, control input matrix and input perturbation vector complete the numerical space allocation in the host machine memory, and together establish the basic linearized mathematical mapping entity of the heterogeneous resource continuous state space model;

[0107] It should be noted that in the above adaptive state manifold evolution model, the diagonal gating matrix... With the system state vector The operator executes dynamic updates of the state components. In engineering implementation, at the beginning of each preset control cycle, the control surface solving module 30 calculates the instantaneous value of the diagonal gating matrix based on the currently sampled system state vector and treats it as a constant matrix within the control cycle, thereby constructing a linear time-varying approximation model. The algebraic Riccati equation solver solves for the optimal feedback gain matrix data within the current cycle based on this linear time-varying approximation model, thereby realizing gain scheduling control of the nonlinear state manifold evolution model.

[0108] Example 3: Generation of Topological Dependency Soft Constraint Matrix Based on Logarithmic Boundary Function

[0109] Please refer to the appendix. Figure 5 This document details the specific implementation of the topological dependency soft constraint matrix generation based on the logarithmic barrier function in the state manifold construction module 20. This embodiment further explains the adaptive state manifold evolution model in step S202, which transforms discrete graph-theoretic topological dependencies into continuously differentiable soft constraint values. Specifically, it includes the following steps:

[0110] In step S2021, the continuous equation mapping unit 22 in the state manifold construction module 20 receives the adjacency matrix output by the computation graph parsing module 10. Based on the adjacency matrix, the continuous equation mapping unit 22 determines all the preceding tensor operators of the target tensor operator. The continuous equation mapping unit 22 reads the completion callback flags of all preceding tensor operators in the execution queue of the underlying hardware by listening to the flow event query interface of the underlying hardware driver. The specific physical form of the completion callback flag is a binary state value. A completion callback flag equal to the value one indicates that the preceding tensor operator has been executed. A completion callback flag equal to the value zero indicates that the preceding tensor operator is in a waiting or executing state. The continuous equation mapping unit 22 performs a weighted summation of the completion callback flags of all preceding tensor operators and divides the weighted summation value by the total number of preceding tensor operators to calculate the completion ratio of the preceding dependencies of the target tensor operator.

[0111] In step S2022, the continuous equation mapping unit 22 constructs a composite soft constraint function based on the inverse of the logarithmic barrier function, according to the proportion of prior dependency completion. The logarithmic barrier function transforms the discrete graph-theoretic topological impedance into a continuously differentiable soft constraint value. The continuous equation mapping unit 22 allocates a diagonal gating matrix in the state vector register space. The continuous equation mapping unit 22 calculates the diagonal elements in the diagonal gating matrix corresponding to each tensor operator. The values ​​of the diagonal elements are determined by the following formula:

[0112]

[0113] In the above formula, the symbols are defined as follows: Characterizing the first in the diagonal gating matrix Line and number The diagonal element at the column intersection; Characterizing the first The prerequisite dependency completion ratio of each tensor operator, wherein the value range of the prerequisite dependency completion ratio is a closed interval including the values ​​of zero and one. Characterizes the obstacle penalty coefficient preset by the hardware system; The tolerance constant for a minimal positive real number is used to prevent numerical singularity anomalies from occurring at the zero point in the hardware logarithmic arithmetic unit; the obstacle penalty coefficient. With the tolerance constant satisfy This ensures that the diagonal elements of the diagonal gating matrix are finite positive real elements;

[0114] In step S2023, the continuous equation mapping unit 22 writes the calculated diagonal elements sequentially into the main diagonal memory address of the diagonal gating matrix. The continuous equation mapping unit 22 assigns the value of zero to the internal elements in the non-main diagonal positions of the diagonal gating matrix. The diagonal gating matrix is ​​stored in the host machine's physical memory in a dense array format for the algebraic Riccati equation solver in the control plane solution module 30 to read and call.

[0115] In step S2024, the continuous equation mapping unit 22 performs matrix multiplication on the diagonal gating matrix and the input perturbation vector. When the completion ratio of the pre-dependency of the target tensor operator is equal to zero, the diagonal elements of the diagonal gating matrix decay and approach zero. The diagonal elements approaching zero minimize the incremental component of the input perturbation vector, thus preventing the system state vector from accumulating enough values ​​to break through the hysteresis state barrier judgment threshold set by the discretization dispatch module 40. This substantially cuts off the path of the target tensor operator into the underlying heterogeneous hardware node at the physical level. The soft constraint mapping mechanism of the logarithmic barrier function ensures the global differentiability of the adaptive state manifold evolution model and avoids the problem of nonlinear differential equation solver crashing and reporting errors caused by directly introducing discrete infinite topological impedance.

[0116] Example 4: Construction of the Functional of Joint Computation-Storage Optimal Control Cost

[0117] Please refer to the appendix. Figure 6 This section provides a detailed explanation of the specific implementation method for constructing the computation-store joint optimal control cost functional in the control surface solution module 30. After the state manifold construction module 20 completes the basic mapping of the continuous state space model, the control surface solution module 30 constructs a cost evaluation function based on this model to guide the optimal scheduling strategy. This includes the following steps:

[0118] Step S301: The control plane solving module 30 initializes the memory space of the cost functional within the physical resource isolation zone. The cost functional is configured as a quadratic integral structure, which includes a quadratic term of the system state vector and a quadratic term of the continuous control input vector. The quadratic term of the system state vector is used to quantify the latency penalty caused by bandwidth contention and task queue backlog. The quadratic term of the continuous control input vector is used to quantify the communication latency and context switching penalty caused by task switching between heterogeneous hardware nodes.

[0119] In step S302, the control plane solving module 30 constructs a state penalty matrix and configures it as a positive semi-definite matrix. The dimension of the state penalty matrix is ​​the same as the dimension of the system state vector. The control plane solving module 30 reads the physical bandwidth upper limit parameter of the peripheral component interconnection bus and the memory access delay constant of each heterogeneous computing node. The control plane solving module 30 maps the reciprocal of the physical bandwidth upper limit parameter and the memory access delay constant to the main diagonal elements of the state penalty matrix. The control plane solving module 30 reads the theoretical total number of floating-point operations and the number of bytes accessed for memory read and write by the tensor operator, and calculates the ratio of the theoretical total number of floating-point operations to the number of bytes accessed for memory read and write. When the ratio is lower than the preset compute-to-memory ratio threshold of the heterogeneous model hybrid inference task priority scheduling and computing power optimization system, the control plane solving module 30 determines that it is currently in the memory-intensive tensor transmission stage. The control plane solving module 30 multiplies the diagonal elements corresponding to the communication resource components in the state penalty matrix by a dynamic penalty coefficient greater than one, thereby increasing the penalty weight for bandwidth contention behavior that deviates from the ideal compute-to-memory affinity state.

[0120] In step S303, the control plane solving module 30 constructs a control penalty matrix and configures it as a positive definite matrix. The dimension of the control penalty matrix is ​​the same as the dimension of the continuous control input vector. The control plane solving module 30 reads the context switching overhead clock cycle value of the underlying operating system kernel of the heterogeneous hardware platform. The control plane solving module 30 maps the context switching overhead clock cycle value proportionally to the main diagonal elements of the control penalty matrix. The main diagonal elements of the control penalty matrix constitute a damping term to suppress the frequent migration of the tensor operator between different hardware nodes, ensuring the smoothness of the continuous scheduling control law and avoiding high-frequency oscillation of instructions in the computing power scheduling system.

[0121] Step S304: The control surface solution module 30, based on the system state vector, state penalty matrix, continuous control input vector, and control penalty matrix, performs infinite time-domain integration on the quadratic form in the continuous time domain to generate a computation-memory joint Hamiltonian cost functional. The mathematical expression of the computation-memory joint Hamiltonian cost functional is as follows:

[0122]

[0123] The symbols in the above formula are defined as follows: The total real value of the output of the computation-store joint Hamiltonian cost functional represents the total cost. Represents a continuous-time state vector; A transpose of a continuous-time state vector; The state penalty matrix represents the positive semidefinite state. Characterizes the continuous control input vector; A transpose vector representing the continuous control input vector; The control penalty matrix represents positive definiteness; Characterizing time integral variables;

[0124] In step S305, the control surface solver module 30 locks and stores the computation-store joint Hamiltonian cost functional parameters in a high-speed static random access memory. The high-speed static random access memory provides a low-latency data addressing channel for the algebraic Riccati equation solver.

[0125] Example 5: Physical Isolation Architecture between Control Plane and Data Plane

[0126] Please refer to the appendix. Figure 7 This paper details the implementation method for constructing a shadow control plane and a high-speed data plane in a heterogeneous model hybrid inference task priority scheduling and computing power optimization system. This embodiment provides resource isolation and a high-speed data path for the control plane solution module 30 and the discretization dispatch module 40 at the system physical deployment level to ensure the parallelism and real-time performance of control law solution and task dispatch. Specifically, it includes the following steps:

[0127] In step S401, the heterogeneous model hybrid inference task priority scheduling and computing power optimization system performs underlying physical resource detection, core binding and basic isolation environment configuration. The heterogeneous model hybrid inference task priority scheduling and computing power optimization system calls the process affinity interface of the operating system to constrain the control plane solving module 30 to the independent physical processor core corresponding to the shadow control plane to perform solving operations. At the same time, the heterogeneous model hybrid inference task priority scheduling and computing power optimization system exclusively binds the discretization dispatch module 40 to the physical processor core corresponding to the high-speed data plane to specifically handle hardware interrupts.

[0128] In step S402, the heterogeneous model hybrid inference task priority scheduling and computing power optimization system establishes a cross-plane global shared memory mapping between the shadow control plane and the high-speed data plane. After the control plane solution module 30 completes the mathematical solution process, it generates an adaptive feedback scheduling control law in the continuous time domain. The control plane solution module 30 parses the adaptive feedback scheduling control law in the continuous time domain into the underlying feedback gain matrix data.

[0129] In step S403, the control plane solving module 30 calls a hardware-level atomic write instruction to directly push the feedback gain matrix data into a pre-initialized lock-free circular buffer in the cross-plane global shared memory. After the feedback gain matrix data is written, the control plane solving module 30 executes a memory barrier instruction and updates the write end index pointer of the lock-free circular buffer. When the discretization dispatch module 40 executes a task in the high-speed data plane, it bypasses the kernel-mode system call channel of the operating system. The discretization dispatch module 40 polls the write end index pointer of the lock-free circular buffer through a physical memory pointer offset instruction. When it is determined that the write end index pointer is inconsistent with the read end index pointer value maintained by the discretization dispatch module 40, the discretization dispatch module 40 reads the latest version of the feedback gain matrix data from the lock-free circular buffer and loads the feedback gain matrix data into a register to generate a discrete scheduling instruction to be sent to the lower layer. After the data reading is completed, the discretization dispatch module 40 synchronously updates the read end index pointer to the write end index pointer value.

[0130] Example 6: Solving the Algebraic Riccati Equation and Generating the Optimal Control Law

[0131] Please refer to the appendix. Figure 8 This document details the specific implementation of the control surface solution module 30 for solving the background of the algebraic Riccati equation and generating the optimal control law. Building upon the computation-memory joint Hamiltonian cost functional constructed in Example 4, this embodiment generates a feedback gain matrix that can guide hardware scheduling through matrix numerical solution. Specifically, it includes the following steps:

[0132] In step S501, the algebraic Riccati equation solver within the control surface solution module 30 extracts the continuous equation mapping parameters and computation-memory joint Hamiltonian cost functional parameters stored in the high-speed static random access memory. The continuous equation mapping parameters include the state decay matrix and the control input matrix, while the computation-memory joint Hamiltonian cost functional parameters include the state penalty matrix and the control penalty matrix. Based on the state decay matrix, the control input matrix, the state penalty matrix, and the control penalty matrix, the control surface solution module 30 constructs the continuous-time algebraic Riccati equation. The mathematical expression of the continuous-time algebraic Riccati equation is as follows:

[0133]

[0134] The symbols in the above formula are defined as follows: Characterizing the state decay matrix; The transpose of the state decay matrix; The matrix representing the unique positive definite symmetric solution to be solved; Characterizes the control input matrix; The transpose of the control input matrix; The inverse matrix representing the control penalty matrix; The state penalty matrix represents the state.

[0135] In step S502, the control surface solution module 30 calls the basic linear algebra subroutine library to perform numerical solution of the continuous-time algebraic Riccati equation. The control surface solution module 30 constructs the Hamiltonian matrix, which is composed of the state decay matrix, the control input matrix, the inverse matrix of the control penalty matrix, and the concatenation of the state penalty matrix. The control surface solution module 30 performs real Schul decomposition on the Hamiltonian matrix to extract the eigenvector subspace corresponding to the eigenvalues ​​with negative real parts of the Hamiltonian matrix. The control surface solution module 30 divides the eigenvector subspace into upper and lower half matrix components according to the row dimension. The control surface solution module 30 performs matrix multiplication of the lower half matrix component and the inverse matrix of the upper half matrix component to calculate the unique positive definite symmetric solution matrix. The unique positive definite symmetric solution matrix represents the optimal cost mapping relationship of the heterogeneous resource continuous state space model in the infinite time domain.

[0136] In step S503, the control surface solution module 30 generates feedback gain matrix data based on the calculated unique positive definite symmetric solution matrix. The control surface solution module 30 performs a series multiplication operation on the inverse matrix of the control penalty matrix, the transpose matrix of the control input matrix, and the unique positive definite symmetric solution matrix to calculate the feedback gain matrix. The feedback gain matrix is ​​the numerical weight coefficient of the conversion of the dispatch rate of each physical state component in the continuous time state vector to the heterogeneous hardware node. The control surface solution module 30 uses the feedback gain matrix to generate an adaptive feedback scheduling control law in the continuous time domain. The adaptive feedback scheduling control law instructs the system to adaptively avoid the continuous task dispatch rate of congested nodes.

[0137] In step S504, the control plane solving module 30 converts the feedback gain matrix into a low-level fixed-point numerical format compatible with the discretization dispatch module 40. The control plane solving module 30 writes the fixed-point numerical format feedback gain matrix data into the lock-free circular buffer between the shadow control plane and the high-speed data plane. By asynchronously executing the dimensionality reduction solution operation of the algebraic Riccati equation in the shadow control plane, the heterogeneous model hybrid inference task priority scheduling and computing power optimization system can provide the optimal physical path mapping data for the dispatch of low-level hardware tasks without occupying the computing resources of the high-speed data plane, thus avoiding bus blocking and computing power idleness.

[0138] Example 7: Design of Discretized Integrator-Accumulator for Continuous Control Flow

[0139] Please refer to the appendix. Figure 9 This document details the specific implementation of the discretized integral accumulation of continuous control flow performed by the discretization dispatch module 40. This embodiment transforms the continuous optimal control law generated by the control surface solver module 30 into a hardware-executable discrete task accumulation, specifically including the following steps:

[0140] In step S601, the discretization dispatch module 40 receives the feedback gain matrix data transmitted by the lock-free ring buffer in the high-speed data plane. The discretization dispatch module 40 obtains the continuous time state vector output in real time by the heterogeneous resource continuous state space model. The discretization dispatch module 40 performs matrix multiplication operation on the negative value of the feedback gain matrix data and the continuous time state vector to calculate the continuous dispatch rate value corresponding to each heterogeneous hardware node. The continuous dispatch rate value represents the data flow rate of the heterogeneous model hybrid inference task priority scheduling and computing power optimization system expected to dispatch tensor operators to specific heterogeneous hardware nodes.

[0141] In step S602, the discretization dispatch module 40 configures an independent integral accumulator logic unit 41 for each heterogeneous hardware node in the heterogeneous hardware platform. The underlying physical implementation of the integral accumulator logic unit 41 is a pre-allocated 64-bit high-precision floating-point register. The integral accumulator logic unit 41 is connected to the hardware clock interrupt signal link of the operating system. In each hardware clock interrupt cycle, the integral accumulator logic unit 41 multiplies the received continuous dispatch rate value of the specific heterogeneous hardware node by the step length of the hardware clock interrupt cycle to obtain the single-step integral increment value. The integral accumulator logic unit 41 performs an addition operation on the single-step integral increment value and the current accumulated value stored in the 64-bit high-precision floating-point register, and writes the calculation result of the addition operation back into the 64-bit high-precision floating-point register to complete the numerical integration and accumulation operation of the continuous control flow in the time dimension.

[0142] In step S603, the discretization dispatch module 40 reads the tensor operator to be processed at the front of the task queue. Based on the total number of theoretical floating-point operations extracted by the operator feature extraction unit 11, the discretization dispatch module 40 determines the hardware computation workload consumed by the tensor operator to be processed. The hardware computation workload represents the minimum computing power threshold required for the tensor operator to be completely dispatched to the underlying hardware execution as an indivisible physical execution unit. The hardware computation workload is discrete and indivisible scalar data, which is used to establish a dimensionally unified comparison benchmark with the continuously accumulated values ​​in the integral accumulator logic unit 41. Through the accumulation operation of the integral accumulator logic unit 41, the continuous-time control instruction is converted into a discrete task accumulation that can be measured by physical hardware, bridging the execution difference between continuous control theory and indivisible physical operators.

[0143] Example 8: Hysteresis State Barrier Determination and Underlying Hardware Discrete Dispatch Mechanism

[0144] Please refer to the appendix. Figure 10This document provides a detailed description of the specific implementation of the hysteresis gradient barrier determination and underlying hardware discrete dispatch in the discretization dispatch module 40. This embodiment, together with Embodiment Seven, constitutes the complete lower-level operation chain of the discretization dispatch module 40, used to introduce a hysteresis determination mechanism based on continuous integral accumulation to prevent high-frequency oscillation migration of tasks between heterogeneous nodes. Specifically, it includes the following steps:

[0145] Step S701: Hysteresis barrier determination unit 42 reads the cross-node communication delay parameter and memory context switching time value of the heterogeneous hardware platform. Hysteresis barrier determination unit 42 adds the cross-node communication delay parameter and memory context switching time value to generate a hysteresis barrier constant. The hysteresis barrier constant quantifies the system overhead generated when the tensor operator to be processed migrates between different heterogeneous hardware nodes.

[0146] In step S702, the hysteresis state barrier determination unit 42 extracts the hardware computation workload value consumed by the tensor operator to be processed, reads the source hardware node identifier where the input data of the tensor operator to be processed is currently located, and compares the source hardware node identifier with the target heterogeneous hardware node identifier corresponding to the integral accumulator logic unit 41. When the source hardware node identifier and the target heterogeneous hardware node identifier are consistent, the hysteresis state barrier determination unit 42 uses the hardware computation workload value as the barrier determination threshold. When the source hardware node identifier and the target heterogeneous hardware node identifier are inconsistent, the hysteresis state barrier determination unit 42 superimposes the hardware computation workload value with the hysteresis barrier constant to generate the barrier determination threshold. By distinguishing the node consistency state, the heterogeneous model hybrid inference task priority scheduling and computing power optimization system prevents the tensor operator to be processed from high-frequency migration oscillation between the central processing unit queue and the graphics processing unit queue.

[0147] In step S703, the discretization dispatch module 40 polls the current accumulated value in the integral accumulator logic unit 41. The discretization dispatch module 40 compares the current accumulated value with the barrier judgment threshold. When the current accumulated value is strictly less than the barrier judgment threshold, the discretization dispatch module 40 keeps the tensor operator to be processed in the waiting queue of the currently allocated hardware node and refuses to dispatch the tensor operator to be processed to the target heterogeneous hardware node. At the same time, the system monitors the dwell time of the tensor operator to be processed in the waiting queue. When the dwell time exceeds the preset starvation judgment threshold, the physical dispatch action is forcibly triggered and the corresponding integral accumulator logic unit 41 is cleared to avoid the starvation phenomenon of the operator staying indefinitely.

[0148] Step S704: When the current accumulated value is greater than or equal to the barrier determination threshold, the discretization dispatch module 40 triggers a physical dispatch operation for the target heterogeneous hardware node. The discretization dispatch module 40 calls the underlying driver application programming interface of the target heterogeneous hardware node. The discretization dispatch module 40 encapsulates the input tensor physical memory pointer, the output tensor physical memory pointer, and the calculation kernel function instruction corresponding to the tensor operator to be processed into an indivisible execution handle. The discretization dispatch module 40 pushes the execution handle to the direct memory access command queue of the target heterogeneous hardware node.

[0149] In step S705, after the execution handle push is completed and the synchronization response signal of the underlying bus is received, the discretization dispatch module 40 performs a state reset update on the integrator accumulator logic unit 41. The discretization dispatch module 40 subtracts the hardware calculation workload value corresponding to the tensor operator to be processed from the current accumulated value of the integrator accumulator logic unit 41 to generate the updated integral residual value. The discretization dispatch module 40 writes the updated integral residual value back into the high-precision floating-point register of the integrator accumulator logic unit 41. Through the value subtraction update operation, the discretization dispatch module 40 eliminates the feedback weight accumulated by the dispatched tensor operator in the integrator accumulator logic unit 41, and realizes the state synchronization between the continuous optimal control flow and the execution state of the underlying physical discrete operator.

Claims

1. A method for priority scheduling and computational power optimization of heterogeneous model hybrid inference tasks, applied to a system for priority scheduling and computational power optimization of heterogeneous model hybrid inference tasks, the system comprising a computation graph analysis module (10), a state manifold construction module (20), a control surface solving module (30), and a discretization dispatch module (40), comprising the following steps: In step S100, the computation graph parsing module (10) receives the multi-model computation graph and establishes the discrete tensor operator states; In step S200, the state manifold construction module (20) generates a diagonal gating matrix based on the graph theory dependency of the tensor operator state, and embeds the tensor operator state mapping into the continuous state space model of the heterogeneous hardware resources to establish an adaptive state manifold evolution model; the control surface solution module (30) calculates the solution matrix within a preset period using an algebraic Riccati equation solver based on the adaptive state manifold evolution model and preset computation-memory joint Hamiltonian cost functional parameters, and outputs an adaptive feedback scheduling control law based on the Riccati equation in the continuous time domain, and parses the adaptive feedback scheduling control law into feedback gain matrix data; the discretization dispatch module (40) converts the feedback gain matrix data into discrete execution instructions that can be recognized by the underlying physical hardware, and dispatches them to the execution queue of the target heterogeneous hardware node to complete a single tensor-level hardware scheduling.

2. The heterogeneous model hybrid inference task priority scheduling and computing power optimization method according to claim 1, characterized in that, In step S100, the process of establishing the discrete tensor operator state includes: Step S110: The computation graph parsing module (10) receives multiple deep learning model inference requests sent concurrently, reads the weight file and network structure description file corresponding to the deep learning model, maps each network layer in the network structure description file to the underlying tensor operator node, maps the data transmission path between network layers to directed edges, assigns independent model bias indexes to each deep learning model, and merges the graph structures of different models into the global computation graph set according to the model bias indexes. Step S120: The operator feature extraction unit (11) traverses the tensor operator nodes in the global computation graph set, extracts the static physical properties of the tensor operators, obtains the input tensor dimension and the output tensor dimension, and reads the data precision format identifier of the tensor operator node; the static physical properties include the input tensor dimension, the output tensor dimension and the data precision format identifier. Step S130: The operator feature extraction unit (11) calculates the theoretical computational amount and memory access bandwidth overhead required for the tensor operator node to be executed on the heterogeneous hardware node based on the static physical properties, and encapsulates the total number of theoretical floating-point operations and the number of bytes accessed for memory reading and writing into an operator feature vector. Step S140: The topology dependency analysis unit (12) establishes the pre-dependency and post-dependency relationships of the tensor operator nodes based on the directed edges in the global computation graph set, and generates an adjacency matrix.

3. The heterogeneous model hybrid inference task priority scheduling and computing power optimization method according to claim 2, characterized in that, in step S200, the process of establishing an adaptive state manifold evolution model includes: Step S210: The underlying physical state acquisition unit (21) acquires the underlying physical performance load data of the heterogeneous hardware platform, obtains the number of threads in the waiting execution state, calculates the number of kernel instructions in the queue and the real-time data transmission throughput. Step S220: The continuous equation mapping unit (22) allocates a continuous-time state vector in the state vector register space and writes the underlying physical performance load data into the memory address of the numerical component corresponding to the continuous-time state vector in sequence. Step S230, the continuous equation mapping unit (22) constructs a state decay matrix, which maps the physical processing capability evolution benchmark of the heterogeneous hardware platform itself; Step S240, the continuous equation mapping unit (22) constructs a control input matrix, and the control input matrix maps the gain transfer path of continuous scheduling instructions to the target physical execution queue; In step S250, the continuous equation mapping unit (22) constructs an input perturbation vector with the same dimension as the total number of global tensor operators based on the operator feature vector. The input perturbation vector quantizes and records the hardware resource computation requests attached to the tensor operator nodes at the execution front in the multi-model hybrid inference task.

4. The heterogeneous model hybrid inference task priority scheduling and computing power optimization method according to claim 3, characterized in that, Step S200, the process of generating the diagonal gating matrix, includes: Step S2021, the continuous equation mapping unit (22) receives the adjacency matrix, determines all the preceding tensor operators of the target tensor operator, and calculates the preceding dependency completion ratio of the target tensor operator by reading the completion callback flag of the preceding tensor operator in the underlying hardware execution queue. In step S2022, the continuous equation mapping unit (22) constructs a composite soft constraint function based on the inverse of the logarithmic barrier function based on the prior dependency completion ratio, and calculates the diagonal elements in the diagonal gating matrix corresponding to each of the target tensor operators. The values ​​of the diagonal elements are determined by the following formula: In the formula, Characterizing the first in the diagonal gating matrix Line and number The diagonal element at the intersection of columns; Characterizing the first The percentage of prerequisite dependencies completeness for each tensor operator; Characterizes the obstacle penalty coefficient preset by the hardware system; Characterized by a minimum positive real number tolerance constant; the obstacle penalty coefficient With the tolerance constant satisfy To ensure that the diagonal elements are finite positive real numbers; when As the value approaches zero, the denominator approaches positive infinity, causing the diagonal elements to decay smoothly and approach zero. In step S2023, the continuous equation mapping unit (22) sequentially writes the calculated diagonal elements into the main diagonal memory address of the diagonal gating matrix, and uniformly assigns the internal elements at non-main diagonal positions to the value of zero. In step S2024, the continuous equation mapping unit (22) performs matrix multiplication on the diagonal gating matrix and the input perturbation vector.

5. The heterogeneous model hybrid inference task priority scheduling and computing power optimization method according to claim 1, characterized in that the preset process of the computation-memory joint Hamiltonian cost functional parameters includes: Step S301, the control surface solving module (30) initializes the memory space of the cost functional, and the cost functional is configured as a quadratic integral structure containing a quadratic term of the system state vector and a quadratic term of the continuous control input vector; Step S302, the control plane solving module (30) constructs a state penalty matrix and configures the state penalty matrix as a positive semi-definite matrix. The state penalty matrix is ​​used to quantify the latency penalty caused by bandwidth contention and computing queue backlog. When constructing the state penalty matrix, the user-preset priority weights carried by the multi-model inference task are extracted. The priority weights are used to perform weighted operations on the main diagonal elements where the physical state components of the corresponding task are located, so that high-priority tasks have a larger latency penalty coefficient. Step S303, the control surface solving module (30) constructs a control penalty matrix and configures the control penalty matrix as a positive definite matrix. The control penalty matrix is ​​used to quantify the communication delay and context switching penalty caused by task switching between heterogeneous hardware nodes. Step S304: The control surface solving module (30) performs infinite time-domain integration on the quadratic form in the continuous time domain based on the system state vector, the state penalty matrix, the continuous control input vector, and the control penalty matrix, generating a computation-memory joint Hamiltonian cost functional. The mathematical expression of the computation-memory joint Hamiltonian cost functional is as follows: In the formula, Represents the total cost as a real number; Characterizes the system state vector; The transpose of the system state vector; The state penalty matrix represents the positive semidefinite state. Characterizes the continuous control input vector; The transpose of the continuous control input vector; The control penalty matrix represents positive definiteness; Characterizing time integral variables; In step S305, the control surface solving module (30) locks and stores the computation-memory joint Hamiltonian cost functional parameters in a high-speed static random access memory.

6. The heterogeneous model hybrid inference task priority scheduling and computing power optimization method according to claim 1, characterized in that the heterogeneous model hybrid inference task priority scheduling and computing power optimization system further includes a physically isolated architecture, and the construction process includes: Step S401: Call the process affinity interface of the operating system to constrain the control plane solving module (30) to the independent physical processor core corresponding to the shadow control plane, and exclusively bind the discretization dispatch module (40) to the physical processor core corresponding to the high-speed data plane. Step S402: Establish a cross-plane global shared memory mapping between the shadow control plane and the high-speed data plane; In step S403, the control surface solving module (30) calls the hardware-level atomic write instruction to directly push the feedback gain matrix data into the pre-initialized lock-free circular buffer in the cross-surface global shared memory and executes the memory barrier instruction. The discretization dispatch module (40) polls the write end index pointer of the lock-free circular buffer through the physical memory pointer offset instruction and reads the latest version of the feedback gain matrix data.

7. The heterogeneous model hybrid inference task priority scheduling and computing power optimization method according to claim 1, characterized in that the process of the control surface solving module (30) outputting the adaptive feedback scheduling control law in the continuous time domain includes: Step S501: The algebraic Riccati equation solver extracts the continuous equation mapping parameters and the computation-memory joint Hamiltonian cost functional parameters to construct a continuous-time algebraic Riccati equation. The mathematical expression of the continuous-time algebraic Riccati equation is as follows: In the formula, Characterizing the state decay matrix; The transpose of the state decay matrix; The matrix representing the unique positive definite symmetric solution to be solved; Characterizes the control input matrix; The transpose of the control input matrix; The inverse matrix representing the control penalty matrix; The state penalty matrix represents the state. Step S502, the control surface solving module (30) constructs the Hamiltonian matrix and performs real Schul decomposition operation, extracts the eigenvector subspace, performs matrix multiplication operation of the lower half matrix component and the inverse matrix of the upper half matrix component, and calculates the solution matrix; Step S503, the control surface solving module (30) performs a series multiplication operation on the inverse matrix of the control penalty matrix, the transpose matrix of the control input matrix and the solution matrix to calculate the feedback gain matrix data, and uses the feedback gain matrix data to generate an adaptive feedback scheduling control law based on the Riccati equation in the continuous time domain. In step S504, the control surface solving module (30) converts the feedback gain matrix data into a low-level fixed-point numerical format compatible with the discretization dispatch module (40).

8. The heterogeneous model hybrid inference task priority scheduling and computing power optimization method according to claim 1, characterized in that the process of the discretization dispatch module (40) converting the discretization dispatch module (40) into discrete execution instructions recognizable by the underlying physical hardware includes: Step S601, the discretization dispatch module (40) receives the feedback gain matrix data transmitted by the lockless ring buffer, performs matrix multiplication operation on the negative value of the feedback gain matrix data and the continuous time state vector, calculates the continuous dispatch rate value corresponding to each heterogeneous hardware node, and performs non-negative saturation truncation processing on the continuous dispatch rate value to ensure that the rate value input to the integral accumulator logic unit (41) is always non-negative; In step S602, the discretization dispatch module (40) configures an independent integral accumulator logic unit (41) for each heterogeneous hardware node. In each hardware clock interrupt cycle, the integral accumulator logic unit (41) multiplies the received continuous dispatch rate value by the step length of the hardware clock interrupt cycle to obtain the single-step integral increment value, and then performs an addition operation with the current accumulated value and writes it back into the register to complete the numerical integral accumulation operation. In step S603, the discretization dispatch module (40) maintains a global tensor operator ready queue, determines the hardware computation workload consumed by the tensor operator to be processed at the front of the global tensor operator ready queue, and performs a non-blocking traversal comparison of the hardware computation workload with the continuously accumulated values ​​in the integral accumulator logic unit (41) of all heterogeneous hardware nodes to establish a dimensionally unified comparison benchmark, so as to lock the heterogeneous hardware node that first meets the barrier determination threshold as the candidate dispatch target.

9. The heterogeneous model hybrid inference task priority scheduling and computing power optimization method according to claim 8, characterized in that the discretization dispatch module (40) includes a hysteresis state barrier determination unit (42), and the determination process includes: Step S701, the hysteresis barrier determination unit (42) reads the cross-node communication delay parameters and memory context switching time values ​​of the heterogeneous hardware platform and performs addition operations to generate the hysteresis barrier constant. Step S702, the hysteresis state barrier determination unit (42) extracts the hardware computation workload value, compares the source hardware node identifier with the target heterogeneous hardware node identifier, and when the source hardware node identifier is consistent with the target heterogeneous hardware node identifier, the hardware computation workload value is used as the barrier determination threshold. When the source hardware node identifier is inconsistent with the target heterogeneous hardware node identifier, the hardware computation workload value is superimposed with the hysteresis barrier constant to generate the barrier determination threshold.

10. The heterogeneous model hybrid inference task priority scheduling and computing power optimization method according to claim 9, characterized in that the process of the discretization dispatch module (40) dispatching to the execution queue of the target heterogeneous hardware node includes: In step S703, the discretization dispatch module (40) compares the current accumulated value in the integral accumulator logic unit (41) with the barrier determination threshold. When the current accumulated value is strictly less than the barrier determination threshold, the discretization dispatch module (40) keeps the tensor operator to be processed in the current hardware node waiting queue and refuses to dispatch the tensor operator to be processed to the target heterogeneous hardware node. At the same time, the system monitors the dwell time of the tensor operator to be processed in the waiting queue. When the dwell time exceeds the preset starvation determination threshold, the physical dispatch action is forcibly triggered and the corresponding integral accumulator logic unit (41) is cleared to avoid the starvation phenomenon of the operator staying indefinitely. Step S704: When the current accumulated value is greater than or equal to the barrier determination threshold, the discretization dispatch module (40) encapsulates the tensor operator to be processed into an indivisible execution handle and pushes it to the direct memory access command queue of the target heterogeneous hardware node. In step S705, after receiving the synchronization response signal from the underlying bus, the discretization dispatch module (40) subtracts the hardware calculation workload value from the current accumulated value to generate the updated integral remaining amount, and writes it back to the integral accumulator logic unit (41) to perform state reset update.

Images