Artificial intelligence model execution method, device, workstation and multi-card cluster system

Abstract

This application relates to an artificial intelligence model execution method, apparatus, workstation, and multi-card cluster system. The method determines the polynomial parameters of each independent computational unit in the artificial intelligence model based on acquired input time-series data; obtains the shared basis matrix of each independent computational unit, and constructs a time-series model operator for each independent computational unit based on the shared basis matrix and polynomial parameters; and executes the artificial intelligence model based on the input time-series data and the time-series model operators of all independent computational units. The artificial intelligence model execution method provided in this application has higher execution efficiency.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an artificial intelligence model execution method, apparatus, workstation, and multi-card cluster system. Background Technology

[0002] As the context length of large models increases and the training scale of multi-GPU clusters expands, system bottlenecks increasingly stem from the memory wall, leading to decreased throughput. For example, traditional Transformer self-attention requires constructing a large-scale intermediate attention matrix for long sequences, with computational and storage overhead increasing quadratically with sequence length. This has led to numerous works on attention mechanisms to accelerate these algorithms. The core idea is to incorporate the data transfer costs of High Bandwidth Memory (HBM) – Static Random-Access Memory (SRAM) – into the algorithm design, rather than simply reducing floating-point operations. However, this results in significant performance degradation. Summary of the Invention

[0003] Therefore, it is necessary to provide an artificial intelligence model execution method, device, workstation, and multi-card cluster system that can improve the execution efficiency of large models, in order to address the above-mentioned technical problems.

[0004] Firstly, this application provides a method for executing an artificial intelligence model, the method comprising:

[0005] Based on the acquired input time series data, determine the polynomial parameters of each independent computational unit in the artificial intelligence model;

[0006] Obtain the shared basis matrix for each independent computational unit, and construct a time series model operator for each independent computational unit based on the shared basis matrix and polynomial parameters;

[0007] The artificial intelligence model is executed based on the input time series data and the time series model operators of all independent computing units.

[0008] In one embodiment, based on the acquired input time-series data, the polynomial parameters and shared basis matrix of each independent computational unit in the artificial intelligence model are determined, including:

[0009] Obtain input timing data;

[0010] Based on the input time series data and the pre-trained coefficient generation network, the polynomial parameters of each independent computation unit are obtained; the polynomial coefficients of each independent computation unit include the polynomial parameters corresponding to each time step of the input time series data.

[0011] In one embodiment, the time-series model operator includes a state transition operator, an input projection operator, and an output projection operator. The polynomial parameters include a first polynomial parameter, a second polynomial parameter, and a third polynomial parameter. For each independent computational unit, a time-series model operator is constructed based on the shared basis matrix and the polynomial parameters, including:

[0012] Construct a state transition operator based on the parameters of the first polynomial and the shared basis matrix;

[0013] Construct the input projection operator based on the second polynomial parameters and the shared basis matrix;

[0014] The output projection operator is constructed based on the parameters of the third polynomial and the shared basis matrix.

[0015] In one embodiment, an artificial intelligence model is executed based on the input timing data and the timing model operators of all independent computing units, including:

[0016] For each independent computing unit, a preset view is generated based on task requirements and time series model operators; the preset view includes a parallel view or a recursive view; the task requirements include at least one of the following: the pattern of the artificial intelligence model, the sequence length of the input time series data, the real-time requirements for executing the artificial intelligence model, and the hardware conditions for executing the artificial intelligence model;

[0017] The artificial intelligence model is executed based on the input time-series data and the preset views of all independent computing units.

[0018] In one embodiment, the method for determining the recursive view includes:

[0019] Based on the time series model operators and the preset recursive formula, the recursive view is determined.

[0020] In one embodiment, the method for determining the parallel views includes:

[0021] Based on the preset state-space duality calculation formula, the recursive view is converted into a parallel view.

[0022] In one embodiment, when the mode of the artificial intelligence model is the inference phase, the artificial intelligence model is executed based on the input time-series data and the time-series model operators of all independent computing units, including:

[0023] For each independent computational unit, a recursive view is determined based on the time-series model operators;

[0024] Substitute the input time series data into the recursive view to determine the first output result;

[0025] The first output results of all independent computing units are fused to obtain the inference result of the artificial intelligence model.

[0026] In one embodiment, when the artificial intelligence model is in the training phase, the artificial intelligence model is executed based on the input time-series data and the time-series model operators of all independent computing units, including:

[0027] For each independent computing unit, a parallel view is determined based on the time-series model operators;

[0028] The second output result is determined based on the input time series data and the parallel view;

[0029] The second output results of all independent computing units are fused to obtain the fused result of the artificial intelligence model, and the training of the artificial intelligence model is completed based on the fused result.

[0030] In one embodiment, the input time-series data includes the input data at the current time and the input data at the next time, and the time-series model operators include the model operators corresponding to the previous time and the model operators corresponding to the current time. The artificial intelligence model is executed based on the input time-series data and the time-series model operators of all independent computing units, including:

[0031] For each independent computing unit, the parallel view of the previous time step is determined based on the model operator corresponding to the previous time step.

[0032] Determine the output result at the current moment based on the input data at the current moment and the parallel view at the previous moment;

[0033] The outputs of all independent computing units at the current time are fused to obtain the fused result at the current time, and the local gradient is calculated based on the fused result at the current time.

[0034] Based on the local gradient, the model operator corresponding to the previous time step, and the input data at the current time step, the model operator corresponding to the current time step is determined, and the model operator corresponding to the current time step is used as the model operator corresponding to the previous time step for the input data at the next time step, so as to determine the output result at the next time step.

[0035] In one embodiment, the output result at the current moment is determined based on the input data at the current moment and the parallel view at the previous moment, including:

[0036] The matrix parameters required to calculate the parallel view of the previous time step are calculated based on the model operator corresponding to the current time step; the matrix parameters include query terms, key terms, and causal masks;

[0037] The causal mask is calculated in the coefficient domain, and the output result at the current time step is determined based on the calculation result, the parallel view of the previous time step, the query item, the key item, and the input data at the current time step.

[0038] In one embodiment, when the AI ​​model is in the inference phase of a multi-GPU cluster, the AI ​​model is executed based on the input time-series data and the time-series model operators of all independent computing units, including:

[0039] Synchronous communication is performed between multiple execution modules to obtain synchronized first target data; the first target data includes polynomial parameters, and the execution module includes multiple independent computation units.

[0040] Based on the synchronized first target data and the shared basis matrix, the time series model operator is obtained;

[0041] For each independent computational unit, a recursive view is determined based on the time-series model operators;

[0042] Substitute the input time series data into the recursive view to determine the third output result;

[0043] The third output results of all independent computing units are fused to obtain the inference results of the artificial intelligence model.

[0044] In one embodiment, when the training phase of the artificial intelligence model is in a multi-GPU cluster mode, training the artificial intelligence model is completed based on the fusion results, including:

[0045] For each execution module, a local gradient is calculated based on the fusion result; each execution module consists of multiple independent computation units.

[0046] Synchronous communication is performed between multiple execution modules on the second target data, and the network parameters of the artificial intelligence model are updated based on the synchronized second target data; the second target data includes local gradients, polynomial parameters, and shared basis matrices.

[0047] Secondly, one embodiment of this application provides a workstation including a central processing unit and at least one graphics processor. The central processing unit is used to acquire input timing data and transmit the input timing data to the at least one graphics processor. The at least one graphics processor is used to perform the steps of the method provided in the first aspect above.

[0048] Thirdly, one embodiment of this application provides a multi-card cluster system, including a scheduling host and multiple computing nodes, each computing node including a central processing unit and multiple graphics processing units; the scheduling host is used to allocate execution tasks of artificial intelligence models to the multiple computing nodes; the central processing unit in each computing node acquires input timing data and transmits the input timing data to the corresponding multiple graphics processing units, so that the graphics processing units perform the steps of the method provided in the first aspect above based on the execution tasks.

[0049] In one embodiment, the graphics processor includes static random access memory, in which polynomial parameters and a shared basis matrix are stored.

[0050] Fourthly, one embodiment of this application provides an artificial intelligence model execution device, the device comprising:

[0051] The determination module is used to determine the polynomial parameters of each independent computational unit in the artificial intelligence model based on the acquired input time series data.

[0052] The module is used to obtain the shared basis matrix for each independent computational unit and to construct a time series model operator for each independent computational unit based on the shared basis matrix and polynomial parameters.

[0053] The execution module is used to execute the artificial intelligence model based on the input time series data and the time series model operators of all independent computing units.

[0054] Fifthly, one embodiment of this application provides a computer device including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method provided in the first aspect above.

[0055] In a sixth aspect, one embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect above.

[0056] In a seventh aspect, one embodiment of this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method provided in the first aspect above.

[0057] The aforementioned artificial intelligence model execution method, apparatus, workstation, and multi-card cluster system determine the polynomial parameters of each independent computing unit in the artificial intelligence model based on the acquired input time-series data; obtain the shared basis matrix of each independent computing unit, and construct a time-series model operator for each independent computing unit based on the shared basis matrix and polynomial parameters; and execute the artificial intelligence model based on the input time-series data and the time-series model operators of all independent computing units. Compared with the use of time-varying dense matrices in traditional technologies for executing artificial intelligence models, this embodiment uses polynomial coefficients and shared basis matrices to represent time-varying dense matrices, compressing a large amount of redundancy into the shared basis matrix and polynomial coefficients. This allows the core computational logic of the artificial intelligence model execution method to reside in registers and shared memory as much as possible, significantly reducing dynamic parameter redundancy. Furthermore, it facilitates caching intermediate results in on-chip storage, reducing data transfer between HBM and on-chip storage, thus lowering HBM memory access bandwidth requirements and effectively alleviating memory wall limitations, thereby improving the efficiency of executing the artificial intelligence model. In addition, separating the polynomial coefficients and shared basis matrix helps reduce the synchronization burden between multiple independent computing units, further improving the efficiency of executing the artificial intelligence model. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a flowchart illustrating the steps of an artificial intelligence model execution method in one embodiment;

[0060] Figure 2 This is a flowchart illustrating the steps of an artificial intelligence model execution method in another embodiment;

[0061] Figure 3 This is a flowchart illustrating the steps of an artificial intelligence model execution method in another embodiment;

[0062] Figure 4 This is a flowchart illustrating the steps of an artificial intelligence model execution method in another embodiment;

[0063] Figure 5 This is a flowchart illustrating the steps of an artificial intelligence model execution method in another embodiment;

[0064] Figure 6 This is a schematic diagram illustrating the equivalent transformation between a recursive view and a parallel view in one embodiment;

[0065] Figure 7 This is a flowchart illustrating the steps of an artificial intelligence model execution method in another embodiment;

[0066] Figure 8 This is a flowchart illustrating the steps of an artificial intelligence model execution method in another embodiment;

[0067] Figure 9 This is a flowchart illustrating the steps of an artificial intelligence model execution method in another embodiment;

[0068] Figure 10 This is a flowchart illustrating the steps of an artificial intelligence model execution method in another embodiment;

[0069] Figure 11 This is a flowchart illustrating the steps of an artificial intelligence model execution method in another embodiment;

[0070] Figure 12 This is a flowchart illustrating the steps of an artificial intelligence model execution method in another embodiment;

[0071] Figure 13 This is a schematic diagram of the structure of an artificial intelligence model execution device in one embodiment;

[0072] Figure 14 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0074] It should be noted that the terms "first," "second," etc., used in this application may be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "at least one" used in this application refers to one or more.

[0075] First, before introducing the technical solutions of the embodiments disclosed in this application, the background technology or technological evolution on which the embodiments of this application are based will be introduced. In the field of artificial intelligence (AI) technology, with the growth of large model contexts and the expansion of multi-card cluster training scale, system bottlenecks increasingly come from the memory wall, namely, the throughput reduction caused by factors such as high-bandwidth memory (HBM) read and write, limited capacity of on-chip static random-access memory (SRAM) / shared memory, cross-card synchronization latency, and memory access conflicts. Traditional Transformer self-attention requires the construction of a large-scale intermediate attention matrix on long sequences, and the computation and storage overhead increases twice with the sequence length. Therefore, a large number of attention mechanisms have emerged to accelerate this process. The core idea is to incorporate the data transfer cost of high-bandwidth memory (HBM-SRAM) into the algorithm design, rather than simply reducing floating-point operations.

[0076] On the other hand, state-space model approaches (especially selective state-space models) offer long-sequence modeling paths with different attention mechanisms. These methods can theoretically achieve linear time complexity through recursive state updates and can be adapted to GPUs / NPUs using parallel algorithms such as parallel scan. Furthermore, State Space Duality (SSD) proposes a way to write a certain type of state-space model as a structured attention mechanism, allowing for parallel views during training and recursive views during inference.

[0077] However, to maintain commutativity and efficient parallelism, existing SSD approaches typically restrict the state transition matrix to a scalar multiplication identity matrix or a minimal diagonal matrix. While such constraints facilitate the construction of efficient computational kernels, they significantly shift inter-channel coupling dynamics outside the recursive body, requiring additional channel mixers or point mixing layers to compensate. For real-world large-scale models or coupled dynamical system tasks, this approach of "separating time dependencies from inter-channel dependencies" presents a structural expression bottleneck: when inter-channel coupling is itself time-varying and entangled with temporal evolution, single-input single-output (SISO) or SISO-like recursions are insufficient.

[0078] Meanwhile, work on high-performance GPU compilers and kernels has shown that abstract algebra is not only suitable for theoretical proofs but can also directly participate in low-level optimization. For example, layout mapping, interleaving / address remapping (Swizzle), and bank conflict avoidance can all be uniformly described through more abstract linear transformations or algebraic objects. However, most of this work focuses on address layout or memory layout itself, rather than algebraically reparameterizing time-series model operators. In other words, some implementations focus more on "how to arrange data" rather than "how to first coordinate the inherently expensive time-varying, dense operators into smaller-dimensional algebraic objects more suitable for hardware execution."

[0079] In recent years, processor architectures for AI and high-performance computing have shown a trend of evolution from "purely general-purpose computing cores" to "cooperative general-purpose cores and dedicated computing units." The representative path is not to rely solely on a unified programmable computing core to handle all workloads, but rather to gradually offload core operators with high-frequency, regular, and reusable characteristics to dedicated hardware units for execution. For AI workloads, capabilities such as matrix multiplication and addition, tensor block operations, mixed-precision computing, and on-chip cache residency are increasingly becoming key factors affecting system throughput and energy efficiency.

[0080] In some implementations, while many deep learning models possess expressive capabilities at the algorithmic level, their core computational graphs often rely on irregular, time-varying, and frequently explicitly materialized dense operators, making it difficult for them to fully adapt to the execution methods of dedicated matrix computation units. Especially in long sequence modeling and multi-GPU cluster training scenarios, performance bottlenecks increasingly stem from data transfer between HBM and on-chip storage, cross-device communication, and execution fragmentation caused by irregular operators, rather than solely from the floating-point computation volume itself.

[0081] The following table shows a comparison of the characteristics of SRAM and DRAM storage media and their distribution in GPU / CPU architectures:

[0082]

[0083] As can be seen from the table above, while DRAM provides space for storing massive amounts of model parameters and intermediate tensors, its high latency and refresh characteristics limit data transfer efficiency. Selective state-space models or attention mechanisms, if involving a large amount of time-dense matrix materialization, will inevitably be limited by the bandwidth bottleneck of DRAM.

[0084] To address the aforementioned issues, some implementation methods improve AI model execution efficiency from the perspectives of hardware units, memory allocation, communication synchronization, and attention operator fusion. This involves system-level optimization based on a given set of dynamic operators. However, the AI ​​model execution method provided in this application improves AI model execution efficiency by altering the mathematical expression and parameter sharing methods of the dynamic operators.

[0085] The artificial intelligence model provided in this application can be applied to a computing device that deploys a general-purpose GPU / NPU, a Tensor Processing Unit (TPU), a Digital Signal Processor (DSP), and a CPU, or it can deploy a dedicated AI accelerator or a models-on-silicon system.

[0086] The technical solution of this application and how the technical solution of this application solves the technical problem are described in detail below with specific embodiments.

[0087] In one embodiment, such as Figure 1 As shown, an artificial intelligence model execution method is provided. This embodiment illustrates the method by applying it to the computing device described above. In this embodiment, the method includes the following steps:

[0088] Step 100: Based on the acquired input time series data, determine the polynomial parameters of each independent computational unit in the artificial intelligence model.

[0089] Input time-series data can be pre-acquired and stored in the shared memory of a computing device. Input time-series data can include current input time-series data and historical context time-series data or intermediate features. The input time-series data is related to the task performed by the artificial intelligence model. The type of input time-series data can be token embedding, continuous time series features, discrete profiles of physical fields, speech / signal sequences, etc. This embodiment does not limit the type of input time-series data, as long as it can be represented as a vector sequence arranged in time.

[0090] An independent computing unit refers to a computing module in an artificial intelligence model that possesses independent computing capabilities, can complete a type of computational logic independently, and does not rely on other units for pre-execution. Multiple independent computing units can complete a type of computational logic in parallel. If the artificial intelligence model includes a multi-head attention mechanism, then the independent computing unit is a single attention head.

[0091] For each independent computational unit in the artificial intelligence model, the computing device determines the polynomial coefficients corresponding to each moment of the independent computational unit based on the input time series data.

[0092] In an optional embodiment, taking a state-space model as an example, assume that the input time series data is... ,in, , This represents the data at time t in the input time series data; the polynomial coefficients at time t of each independent computation unit include three coefficients, which are represented as follows: , , The term p represents the degree of the polynomial coefficients. In practice, it is usually chosen to be of a lower order to control overhead, but it is not required to choose a fixed degree.

[0093] Step 110: Obtain the shared basis matrix for each independent computational unit, and construct a time series model operator for each independent computational unit based on the shared basis matrix and polynomial parameters.

[0094] The shared basis matrix can be pre-acquired and stored in the computing device. It is understood that the shared basis matrix can be stored in the SRAM of the computing device.

[0095] In an optional embodiment, the method for determining the shared basis matrix includes: acquiring training samples, which include input time-series data samples; training an initial network model based on the training samples to obtain the shared basis matrix. The initial network model includes the structure of the initial basis matrix. The structure of the initial basis matrix can be a fully dense matrix, a diagonal plus low-rank matrix (e.g., ... Other structured matrices that facilitate numerical stability and execution efficiency, such as adjoint matrices, block diagonal matrices, block cyclic matrices, and block Toeplitz matrices, are acceptable. This embodiment does not limit the specific type of the initial basis matrix structure, as long as it achieves its function.

[0096] After obtaining the shared basis matrix and polynomial coefficients for each independent computing unit, the computing device can construct a time-series model operator based on the shared basis matrix and polynomial coefficients. It can be understood that the time-series model operator in an artificial intelligence model is represented in the form of a shared basis matrix plus polynomial coefficients. Specifically, the computing device can perform matrix polynomial parameterization on the shared basis matrix based on the shared basis matrix and polynomial coefficients to obtain the time-series model operator.

[0097] In an optional embodiment, the shared basis matrix of each independent computational unit can be represented as: .

[0098] Step 120: Execute the artificial intelligence model based on the input time series data and the time series model operators of all independent computing units.

[0099] Executing an artificial intelligence (AI) model includes training or inference on the AI ​​model. When executing an AI model, each independent computing unit has a corresponding execution task. The computing device can obtain the temporal model operators for all independent computing units in the AI ​​model in the manner described above. Each independent computing unit executes its corresponding task based on the temporal model operators and the input temporal data, thereby executing the AI ​​model based on the input temporal data and the temporal model operators of all independent computing units. The specific process of executing the AI ​​model based on the input temporal data and the temporal model operators of all independent computing units depends on the type of AI model being executed; this embodiment does not limit this, as long as the functionality is achieved.

[0100] This application provides an artificial intelligence model execution method. The method determines the polynomial parameters of each independent computational unit in the artificial intelligence model based on acquired input time-series data; obtains the shared basis matrix of each independent computational unit; and constructs a time-series model operator for each independent computational unit based on the shared basis matrix and polynomial parameters. The method then executes the artificial intelligence model based on the input time-series data and the time-series model operators of all independent computational units. Compared to the use of time-varying dense matrices in traditional artificial intelligence model execution, this embodiment uses polynomial coefficients and a shared basis matrix to represent the time-varying dense matrix, compressing a large amount of redundancy into the shared basis matrix and polynomial coefficients. This allows the core computational logic of the artificial intelligence model execution method to reside in registers and shared memory as much as possible, significantly reducing dynamic parameter redundancy. Furthermore, it facilitates caching intermediate results in on-chip storage, reducing data transfer between the HBM and on-chip storage, thus lowering the HBM memory access bandwidth requirements and effectively alleviating memory wall limitations, thereby improving the efficiency of executing the artificial intelligence model. Additionally, separating the polynomial coefficients and the shared basis matrix helps reduce the synchronization burden between multiple independent computational units, further improving the efficiency of executing the artificial intelligence model.

[0101] Furthermore, by using polynomial coefficients and shared basis matrices to represent time-varying dense matrices, it is easier to map tasks executed by independent computing units to a unified parameterization and execution mechanism at the GPU / NPU memory access level. Moreover, the artificial intelligence model execution method provided in this application has hardware optimizations for dedicated matrix multiply-accumulate units, on-chip caching, and mixed-precision execution, making it applicable to existing GPUs / NPUs. This allows the algorithm structure to work in tandem with the hardware execution structure, improving the practicality of the artificial intelligence model execution method.

[0102] When the primary execution target shifts from large matrices to small coefficient blocks, localized block masking sub-blocks, or grouped convolution buffers, memory layout optimization becomes more effective. By performing operations such as swizzling / address remapping, padding / alignment padding, bank permutation, and linear layout mapping on submatrices, intermediate result buffers, or shared memory layouts during AI model execution, the layout of the GPU in computing devices can be optimized. This reduces the probability of bank conflicts and pipeline blockages, minimizes shared memory conflicts, and ultimately improves the efficiency of executing AI models.

[0103] In an alternative embodiment, the Cayley-Hamilton theorem is used to fold the powers of the polynomial coefficients above the state dimension back onto a lower-order polynomial basis, avoiding explicit maintenance of excessively high powers. The lower-order polynomial basis can be a monomial basis, an orthogonal polynomial basis (e.g., Chebyshev, Legendre), a truncated basis based on minimal polynomials, or another function expansion basis.

[0104] In an optional embodiment, positional encoding can be added to the input time series data to distinguish the input data at different times within the input time series data. Alternatively, absolute position, relative position, periodic position features, or other time index features can be injected into the coefficient generation network.

[0105] In one embodiment, such as Figure 2 As shown, this involves an implementation method for determining the polynomial parameters of each independent computational unit in an artificial intelligence model based on acquired input time-series data. The steps of this implementation method include:

[0106] Step 200: Obtain input timing data.

[0107] The computing device contains a CPU. The input timing data can be input by the user to the CPU, which is then retrieved by the GPU; alternatively, it can be pre-stored in the shared memory of the computing device, from which the GPU retrieves it. The computing device acquires the input timing data when the model execution method is manually executed.

[0108] Step 210: Generate a network based on the input time series data and the pre-trained coefficients to obtain the polynomial parameters of each independent computation unit; the polynomial coefficients of each independent computation unit include the polynomial parameters corresponding to each time step of the input time series data.

[0109] The coefficient generation network can be pre-trained and stored in the on-chip memory of the computing device. After receiving input time-series data, the GPU in the computing device feeds the input time-series data into the pre-trained coefficient generation network. The coefficient generation network generates polynomial parameters for each independent computing unit. The polynomial parameters include multiple polynomial coefficients, which correspond to each time step in the input time-series data. It can be understood that the computing device can generate polynomial coefficients for each time step and each independent computing unit based on the input time-series data. This embodiment does not limit the structure or type of the coefficient generation network, as long as its functionality is achieved.

[0110] In an alternative embodiment, the coefficient generation network can be a linear layer, a multi-layer perceptron (MLP), a small hyper-network, a convolutional subnetwork, a gated network, a lightweight attention mechanism, a hybrid expert network, etc.

[0111] In an optional embodiment, the training method for the coefficient generation network includes: acquiring training samples; the training samples include input time-series data samples and a matrix polynomial structure; and training an initial network model using the training samples based on the backpropagation algorithm to obtain the coefficient generation network. The matrix polynomial structure refers to the structure corresponding to the polynomial parameters, that is, the representation of the polynomial parameters obtained through the coefficient generation network.

[0112] In this embodiment, after acquiring the input time series data, the polynomial parameters of each independent computation unit are obtained based on the input time series data and the pre-trained coefficient generation network. This method of obtaining polynomial parameters is fast and easy to implement, thereby improving the efficiency of subsequent execution of artificial intelligence models based on polynomial parameters and shared basis matrices.

[0113] In one embodiment, when the artificial intelligence model is a state-space model, the constructed temporal model operators include a state transition operator, an input projection operator, and an output projection operator. The polynomial parameters include a first polynomial parameter, a second polynomial parameter, and a third polynomial parameter. The first polynomial parameter corresponds to the state transition operator, the second polynomial parameter corresponds to the input projection operator, and the third polynomial parameter corresponds to the output projection operator. The same shared basis matrix is ​​used for each independent computational unit. Based on this, such as... Figure 3 As shown, this relates to an implementation method for constructing a time-series model operator for each independent computational unit based on a shared basis matrix and polynomial parameters. The steps of this implementation method include:

[0114] Step 300: Construct the state transition operator based on the parameters of the first polynomial and the shared basis matrix.

[0115] The computing device constructs the state transition operator in the time-series model operator based on the first polynomial parameter and the shared basis matrix. Understandably, the computing device constructs the state transition operator by calculating the product of the first polynomial parameter and the shared basis matrix.

[0116] In an optional embodiment, for each independent computational unit, the state transition operator at time t can be expressed as: .

[0117] Step 310: Construct the input projection operator based on the second polynomial parameters and the shared basis matrix.

[0118] The computing device constructs the input projection operator in the time-series model operator based on the second polynomial parameter in the polynomial parameters and the shared basis matrix. Understandably, the computing device constructs the input projection operator by calculating the product of the second polynomial parameter and the shared basis matrix.

[0119] In an optional embodiment, for each independent computational unit, the input projection operator at time t can be expressed as: .

[0120] Step 320: Construct the output projection operator based on the parameters of the third polynomial and the shared basis matrix.

[0121] The computing device constructs the output projection operator in the time-series model operator based on the third polynomial parameter and the shared basis matrix. Understandably, the computing device constructs the output projection operator by calculating the product of the third polynomial parameter and the shared basis matrix.

[0122] In an optional embodiment, for each independent computational unit, the output projection operator at time t can be expressed as: .

[0123] In this embodiment, the generated polynomial parameters include a first polynomial parameter, a second polynomial parameter, and a third polynomial parameter. The constructed time-series model operators include a state transition operator, an input projection operator, and an output projection operator. The state transition operator can be constructed based on the first polynomial parameter and the shared basis matrix, the input projection operator can be constructed based on the second polynomial parameter and the shared basis matrix, and the output projection operator can be constructed based on the third polynomial parameter and the shared basis matrix. This method of obtaining the state transition operator, input projection operator, and output projection operator is fast and easy to implement. Furthermore, the state transition operator, input projection operator, and output projection operator obtained in this way lie in the same algebraic coordinate system and are commutative, providing conditions for subsequent state space duality and parallel execution.

[0124] In one embodiment, such as Figure 4As shown, this relates to an implementation method for executing an artificial intelligence model based on input time-series data and time-series model operators of all independent computational units. The steps of this implementation method include:

[0125] Step 400: For each independent computing unit, generate a preset view based on task requirements and time series model operators; the preset view includes a parallel view or a recursive view; the task requirements include at least one of the following: the pattern of the artificial intelligence model, the sequence length of the input time series data, the real-time requirements for executing the artificial intelligence model, and the hardware conditions for executing the artificial intelligence model.

[0126] Parallel views and recursive views represent two execution paths in artificial intelligence models. A recursive view computes sequentially step-by-step over time, using the hidden state from the previous time step, combined with the current input, to calculate the current state, and so on. A parallel view unfolds the entire sequence at once, completing the computation in parallel.

[0127] Task requirements refer to the computational task allocation for each independent computation when executing an artificial intelligence (AI) model. AI models include training and inference modes. Typically, the default view for training mode is a parallel view, while the default view for inference mode is a recursive view. Longer input time-series data sequences correspond to a recursive view, while shorter sequences correspond to a parallel view. Higher real-time performance requirements for AI model execution necessitate a parallel view, while lower real-time requirements favor a recursive view. Hardware requirements for executing an AI model include at least one of the following: available video memory, hardware generation, and load balancing. Ample video memory corresponds to a parallel view, while limited video memory corresponds to a recursive view. Different hardware generations correspond to different types of default views. Load balancing results in more tasks being performed by the computing device, leading to a recursive view; conversely, less tasks are handled by the device, resulting in a parallel view.

[0128] For each independent computing unit, the computing device selects a preset view type from the parallel view and the recursive view according to the task requirements, and generates the final preset view according to the time series model operator.

[0129] Step 410: Execute the artificial intelligence model based on the input time series data and the preset views of all independent computing units.

[0130] For each independent computing unit, the output result of that independent computing unit is obtained based on the input time-series data and the preset view. The execution of the artificial intelligence model can be achieved based on the output results of all independent computing units. This embodiment does not limit the specific method of executing the artificial intelligence model based on the input time-series data and the preset view of all independent computing units, as long as its function can be achieved.

[0131] In this embodiment, a preset view is generated based on task requirements and time series model operators; the artificial intelligence model is executed based on the input time series data and the preset views of all independent computing units. In this way, the corresponding preset view can be called to execute the artificial intelligence model for different task requirements, which can improve the flexibility and practicality of the artificial intelligence model execution method.

[0132] In one embodiment, such as Figure 5 As shown, this relates to one implementation of determining recursive views and parallel views, wherein the method for determining the recursive view includes:

[0133] Step 500: Determine the recursive view based on the time series model operators and the preset recursive formula.

[0134] The preset recursive formula includes a state transition matrix, an input projection matrix, and an output projection matrix. After obtaining the time series model operator, the computing device replaces the corresponding state transition matrix, input projection matrix, and output projection matrix in the preset recursive formula with the state transition operator, input projection operator, and output projection operator in the time series model operator to obtain the recursive view.

[0135] In an optional embodiment, the preset recursive formula can be expressed as: , Based on the time-series model operators and the pre-defined recursive formula, the recursive view of the h-th independent computational unit at time t can be represented as follows: , Recursive views are suitable for autoregressive inference, streaming inference, or scenarios that require explicit maintenance of hidden state caches. This eliminates the need to frequently load large time-varying matrices from HBM during inference; instead, time-series model operators can be constructed in on-chip storage by modulating polynomial coefficients, thereby reducing input / output latency.

[0136] Methods for determining parallel views include:

[0137] Step 510: Convert the recursive view into a parallel view according to the preset state space duality calculation formula.

[0138] A schematic diagram illustrating the equivalent transformation between the dual recursive view and the parallel view in the state space is shown below. Figure 6 As shown, Figure 6The edges in the diagram represent the stepwise recursive evolution of the state-space model over time, where the state update is controlled by a time-varying matrix. , and . Figure 6 The right side of the middle section shows a parallel view, where the commutativity introduced by matrix polynomial parameterization (a temporal model operator) maps recursive computation equivalently to a structured attention mechanism. .

[0139] After obtaining the recursive view, the computing device can convert the recursive view into a parallel view according to the preset state space duality calculation formula.

[0140] In an optional embodiment, the preset state-space duality calculation formula can be expressed as: According to this formula, the parallel view of the h-th independent computing unit can be represented as: ,in, It is determined by each moment. Stacked together, It is determined by each moment. Stacked together, Let the matrix be a causal mask, where the first... Each block is from time [time]. At the time The accumulation of state transition operators, This represents a block-wise matrix multiplicative combination. This indicates that the block matrix applies an effect to the entire sequence matrix.

[0141] In an optional embodiment, the atomic interactions between each time step in this application are D×D block matrices, so the rotations, algorithms, and remixing between channels can occur directly within the ladder recursive dual body. This matrix value characteristic enables the artificial intelligence model to capture complex time-varying dynamic couplings during execution. Typically, the parameter size of the traditional Transformer is several times that of the aforementioned artificial intelligence model, achieving extremely high parameter efficiency.

[0142] In this embodiment, by utilizing the commutativity of time-series model operators, a recursive view can be determined based on the time-series model operators and a preset recursive formula. The recursive view can be converted into a parallel view based on a preset state-space dual calculation formula. This allows for the coupling of cross-channel features within the recursive entity while preserving both parallel training and recursive inference modes.

[0143] In one embodiment, when the mode of the artificial intelligence model is the inference phase, such as Figure 7As shown, this relates to an implementation method for executing an artificial intelligence model based on input time-series data and time-series model operators of all independent computational units. The steps of this implementation method include:

[0144] Step 700: For each independent computational unit, determine the recursive view based on the time series model operator.

[0145] When the AI ​​model is in the inference phase, the corresponding preset view type is a recursive view type. For each independent computing unit, the computing device can determine the final recursive view based on the temporal model operator. A description of determining the final recursive view based on the temporal model operator can be found in the detailed description of the above embodiments, and will not be repeated here.

[0146] Step 710: Substitute the input timing data into the recursive view to determine the first output result.

[0147] After obtaining the recursive view, the computing device can obtain the first output result of the independent computing unit by substituting the input timing data into the recursive view.

[0148] Step 720: Merge the first output results of all independent computing units to obtain the inference results of the artificial intelligence model.

[0149] After obtaining the first output results from all independent computing units, the computing device fuses these first output results to obtain the inference result of the artificial intelligence model. It can be understood that the computing device obtains the inference result of the artificial intelligence model at time t by fusing the first output results of all independent computing units at time t. The computing device can cache the inference result at time t for use in the next iteration. This embodiment does not limit the specific method for fusing the first output results of all independent computing units, as long as the function is achieved.

[0150] In an alternative embodiment, the fusion of the first outputs of all independent computing units can be a linear mapping, gated fusion, residual superposition, hybrid expert fusion, or other forms.

[0151] In this embodiment, for each independent computing unit, a recursive view is determined based on the time-series model operator; the input time-series data is substituted into the recursive view to determine the first output result; the first output results of all independent computing units are fused to obtain the inference result of the artificial intelligence model. When the mode of the artificial intelligence model is in the inference stage, inference of the artificial intelligence model can be performed based on the recursive view and the input time-series data. This eliminates the need to reconstruct the entire sequence parallel graph, requiring only the maintenance of a small amount of state cache and shared basis matrix to achieve incremental inference. Compared with the scheme of explicitly reconstructing a large number of intermediate tensors at each step, the artificial intelligence model execution method is more suitable for low-latency scenarios, thereby improving the practicality of the artificial intelligence model execution method.

[0152] In one embodiment, when the mode of the artificial intelligence model is in the training phase, such as Figure 8 As shown, this relates to an implementation method for executing an artificial intelligence model based on input time-series data and time-series model operators of all independent computational units. The steps of this implementation method include:

[0153] Step 800: For each independent computing unit, determine the parallel view based on the time-series model operator.

[0154] When the artificial intelligence model is in the training phase, the corresponding preset view type is the parallel view type. For each independent computing unit, the computing device can determine the final parallel view based on the time-series model operators. A description of determining the final parallel view based on the time-series model operators can be found in the detailed description of the above embodiments, and will not be repeated here.

[0155] Step 810: Determine the second output result based on the input timing data and parallel view.

[0156] The computing device substitutes the input timing input into the parallel view to obtain the second output result of the independent computing unit.

[0157] Step 820: Fuse the second output results of all independent computing units to obtain the fusion result of the artificial intelligence model, and complete the training of the artificial intelligence model based on the fusion result.

[0158] After obtaining the second output results of all independent computing units, the computing device fuses these second output results to complete the current iteration training of the artificial intelligence model. It can be understood that the computing device fuses the first output results of all independent computing units at time t to obtain the fused result of the artificial intelligence model at time t. The computing device can cache the fused result at time t for use in the next iteration. This embodiment does not limit the specific method for fusing the first output results of all independent computing units, as long as the function is achieved.

[0159] In this embodiment, for each independent computing unit, a parallel view is determined based on the time series model operator; by substituting the input time series data into the parallel view, a second output result of each independent computing unit can be obtained; the second output results of all independent computing units are fused to obtain the fusion result of the artificial intelligence model; and the training of the artificial intelligence model is completed based on the fusion result, which can improve the practicality of the artificial intelligence model training method.

[0160] In one embodiment, the input time-series data includes input data from different times, namely, the input data at the current time and the input data at the next time. The corresponding time-series model operators include the model operator corresponding to the previous time and the model operator corresponding to the current time. Based on this, as follows... Figure 9 As shown, this involves a specific implementation method for executing an artificial intelligence model based on input time-series data and time-series model operators of all independent computing units. The steps of this implementation method include:

[0161] Step 900: For each independent computing unit, determine the parallel view of the previous time step based on the model operator corresponding to the previous time step.

[0162] For each independent computing unit, the computing device determines the parallel view corresponding to the previous time step based on the model operator of the previous time step. The description of determining the parallel view corresponding to the previous time step based on the model operator of the previous time step can be found in the above embodiment. The specific description of determining the recursive view based on the time-series model operator and the preset recursive formula, and converting the recursive view into a parallel view based on the preset state-space dual calculation formula, will not be repeated here.

[0163] Step 910: Determine the output result for the current moment based on the input data at the current moment and the parallel view at the previous moment.

[0164] For each independent computing unit, after obtaining the parallel view of the previous moment, the computing device substitutes the input data of the current moment into the parallel view of the previous moment to obtain the output result of the independent computing unit at the current moment.

[0165] Step 920: Fuse the output results of all independent computing units at the current time to obtain the fused result at the current time, and calculate the local gradient based on the fused result at the current time.

[0166] After obtaining the output results of all independent computing units at the current moment, the computing device performs a fusion process on the output results of all independent computing units at the current moment to obtain the fused result at the current moment. The description of fusing the output results at the current moment can be found in the detailed description of the above embodiments, and will not be repeated here.

[0167] After obtaining the fusion result at the current moment, the computing device calculates the local gradient based on the fusion result at the current moment, that is, it calculates the contribution of the loss at the current moment to the network parameters in the artificial intelligence model. This embodiment does not limit the specific method for calculating the local gradient, as long as it can achieve its function.

[0168] Step 930: Based on the local gradient, the model operator corresponding to the previous time step, and the input data of the current time step, determine the model operator corresponding to the current time step, and use the model operator corresponding to the current time step as the model operator corresponding to the previous time step for the input data of the next time step, so as to determine the output result of the next time step.

[0169] The computing device can calculate the model operator for the current time step based on the calculated local gradient, the model operator corresponding to the previous time step, and the input data of the current time step. In other words, the computing device updates the model operator corresponding to the previous time step using the local gradient and the input data of the current time step to obtain the model operator for the current time step. Specifically, the polynomial parameters of the previous time step are updated, and the model operator for the current time step is determined based on the updated polynomial parameters and the shared basis matrix.

[0170] After obtaining the model operator corresponding to the current time step, the computing device uses this model operator as the model operator corresponding to the previous time step for the input data of the next time step, determines the output result of the next time step, and fuses the output results of all independent computing units for the next time step to obtain the fused result of the next time step. Repeating the above process enables the execution of the artificial intelligence model until an iteration stopping condition is met. The iteration stopping condition can include reaching a preset threshold number of iterations, or local gradient convergence, etc.

[0171] This embodiment describes the specific process of executing an artificial intelligence model. This process is quick and easy to implement, and can improve the practicality of the artificial intelligence model execution method.

[0172] In one embodiment, such as Figure 10As shown, this describes an implementation method for determining the output result at the current time step based on the input data at the current time step and the parallel view at the previous time step. The steps of this implementation method include:

[0173] Step 1001: Calculate the matrix parameters required for the parallel view of the previous time step based on the model operator corresponding to the current time step; the matrix parameters include query terms, key terms, and causal masks.

[0174] Based on the commutativity of the model operators corresponding to the current time step, and using the model operators and input data at the current time step, the computing device can calculate the query terms, key terms, and causal mask in the matrix parameters required for the parallel view at the previous time step. In other words, the computing device can calculate the query terms required for the parallel view at the previous time step based on the output projection operators in the model operators corresponding to the current time step; that is, stacking the output projection operators from all time steps before the current time step yields the query terms. Similarly, it can calculate the key terms required for the parallel view at the previous time step based on the input projection operators in the model operators corresponding to the current time step; and it can calculate the causal mask required for the parallel view at the previous time step based on the state transition operators in the model operators corresponding to the current time step; that is, accumulating the state transition operators from all time steps before the current time step yields the causal mask.

[0175] In an optional embodiment, the time-series model operator (the model operator at the current time step) can be represented as:

[0176]

[0177] Among them, the cumulative state transition operators from time 0 to time 1 are: The cumulative state transition operator from time 1 to time 2 is: By analogy, the causal mask corresponding to the parallel view can be obtained.

[0178] Step 1002: Calculate the causal mask in the coefficient domain, and determine the output result at the current time based on the calculation result, the parallel view of the previous time step, the query item, the key item, and the input data at the current time step.

[0179] In the calculation of the causal mask in the matrix parameters, directly performing matrix multiplication for the prefix transition product would lead to high complexity for long sequences. Therefore, in this embodiment, it is transformed to the coefficient domain, that is... Turn to ,in, This corresponds to a scalar polynomial. The causal mask can then be calculated in the coefficient field to obtain the result. This embodiment does not limit the specific method for calculating the causal mask in the coefficient field, as long as the function can be achieved.

[0180] In an alternative embodiment, the Fast Fourier Transform (FFT) can be used, or divide-and-conquer convolution, Karatsuba-like methods, Winograd-like methods, lookup table methods, or other fast multiplication methods supported by hardware primitives can be used.

[0181] After obtaining the calculation result of the causal mask, the computing device substitutes the calculation result, query item, key item and the input data of the current moment into the parallel view of the previous moment to obtain the output result of the independent computing unit at the current moment.

[0182] In this embodiment, the matrix parameters required for the parallel view of the previous time step are calculated based on the model operator corresponding to the current time step. The matrix parameters include query terms, key terms, and a causal mask. The causal mask is calculated in the coefficient domain, and the output result for the current time step is determined based on the calculation result, the parallel view of the previous time step, the query terms, the key terms, and the input data of the current time step. This method of calculating the causal mask in the coefficient domain reduces the size of intermediate objects, converts polynomial multiplication to convolution, facilitates parallel scanning / tree reduction, and is more suitable for temporary storage in on-chip memory, thus improving the efficiency of executing artificial intelligence models. Furthermore, calculating the causal mask in the coefficient domain avoids… Dense matrix multiplication is then performed. The coefficient convolution significantly reduces the queuing overhead of independent computing units.

[0183] In an optional embodiment, for each independent computational unit, there exists a mapping from the scalar polynomial space to the matrix subalgebra: Therefore, for each independent computational unit, objects that need to be processed in the matrix domain can be processed in the coefficient domain first, and then evaluated back to the matrix domain when needed. This can reduce the volume of intermediate objects, convert polynomial multiplication into convolution, make it easier to utilize parallel scanning / tree reduction, and is more suitable for temporary storage in on-chip memory.

[0184] In one embodiment, when the AI ​​model operates in a multi-GPU cluster inference phase, such as Figure 11 As shown, this relates to an implementation method for executing an artificial intelligence model based on input time-series data and time-series model operators of all independent computational units. The steps of this implementation method include:

[0185] Step 1101: Perform synchronous communication between multiple execution modules on the first target data to obtain synchronized first target data; the first target data includes polynomial parameters, and the execution module includes multiple independent calculation units.

[0186] A multi-GPU cluster refers to a computing cluster composed of multiple GPUs and multiple execution modules (servers or terminals). Each execution module includes multiple GPUs, meaning each execution module contains multiple independent computing units. Within each execution module, the shared basis matrix of each independent computing unit is pre-acquired and stored, while the polynomial parameters corresponding to each independent computing unit are updated in real time. Synchronization communication of these polynomial parameters is then performed between multiple execution modules. This involves synchronizing the polynomial parameters of each independent computing unit in each execution module to the independent computing units in other execution modules. Each execution module then receives the synchronized first target data, namely the polynomial parameters of each independent computing unit in other execution modules.

[0187] Step 1102: Based on the synchronized first target data and the shared basis matrix, obtain the time series model operator.

[0188] For each independent computing unit within each execution module, the computing device determines the timing model operator for each independent computing unit based on the synchronized first target data and the shared basis matrix. The description of determining the timing model operator for each independent computing unit based on the synchronized first target data and the shared basis matrix can be found in the detailed description of constructing the timing model operator based on the shared basis matrix and polynomial parameters in the above embodiments, and will not be repeated here.

[0189] Step 1103: For each independent computational unit, determine the recursive view based on the time series model operator.

[0190] Step 1104: Substitute the input timing data into the recursive view to determine the third output result.

[0191] Step 1105: Merge the third output results of all independent computing units to obtain the inference results of the artificial intelligence model.

[0192] For the descriptions of steps 1103, 1104, and 1105, please refer to the above. Figure 7 The specific descriptions of the embodiments shown will not be repeated here.

[0193] In this embodiment, during the inference phase of the AI ​​model in a multi-GPU cluster mode, the first target data is first synchronously communicated between multiple execution modules to obtain synchronized first target data. The first target data includes polynomial parameters, and the execution modules include multiple independent computing units. Based on the synchronized first target data and the shared basis matrix, a time-series model operator is obtained. For each independent computing unit, a recursive view is determined based on the time-series model operator. The input time-series data is substituted into the recursive view to determine the third output result. The third output results of all independent computing units are fused to obtain the inference result of the AI ​​model. In this way, the data for synchronous communication between multiple execution modules only includes polynomial parameters, which can reduce the amount of communication data, mitigate the throughput loss caused by cross-execution module synchronization, reduce HBM read / write and buffer pressure, and reduce the startup cost of computing cores, thereby improving the execution efficiency of the AI ​​model.

[0194] In one embodiment, when the training phase of the artificial intelligence model is in the mode of a multi-GPU cluster, such as Figure 12 As shown, this relates to an implementation method for training an artificial intelligence model based on the fusion results. The steps of this implementation method include:

[0195] Step 1201: For each execution module, calculate the local gradient based on the fusion result; the execution module includes multiple independent calculation units.

[0196] For a description of multi-GPU clusters, please refer to the detailed description in the above embodiments, which will not be repeated here. For each execution module, a local gradient is calculated based on the fusion result of multiple computing units within that execution module. Specifically, the computing device can calculate a loss function based on the fusion result and then calculate the local gradient based on the loss function.

[0197] Step 1202: Perform synchronous communication between multiple execution modules on the second target data, and update the network parameters of the artificial intelligence model based on the synchronized second target data; the second target data includes local gradients, polynomial parameters and shared basis matrices.

[0198] After obtaining the local gradient corresponding to each execution module, the computing device synchronizes the local gradient, polynomial parameters, and shared basis matrix of that execution module to other execution modules. Based on the synchronized second target data, the network parameters of the artificial intelligence model are updated according to the backpropagation calculation, thereby realizing the training iteration of the artificial intelligence model.

[0199] In an optional embodiment, the second target data further includes: a low-dimensional summary derived from polynomial parameters, or quantization / sparse / grouping of polynomial parameters, quantization / sparse / grouping of shared basis matrices, etc.

[0200] In this embodiment, for each execution module, a local gradient is calculated based on the fusion result; each execution module includes multiple independent computing units; synchronous communication is performed between multiple execution modules on the second target data, and the network parameters of the artificial intelligence model are updated based on the synchronized second target data. Compared with the traditional technique of synchronizing a complete dense matrix, the synchronized second target data between multiple execution modules only includes local gradients, polynomial parameters, and shared basis matrices, which can reduce the bandwidth occupation across execution modules and reduce the amount of communication data. The amount of communication data was reduced to a magnitude of 100. Even in the case of low-order polynomials, it can be further reduced to This reduces throughput loss caused by synchronization across execution modules, lowers HBM read / write pressure and buffer pressure, and reduces startup costs of computing cores, thereby improving the execution efficiency of artificial intelligence models.

[0201] In an optional embodiment, the computing device includes a coefficient generation module, a shared basis matrix buffer module, a coefficient domain operation module, a matrix block computation module, a mode scheduling module, and an output fusion module. The coefficient generation module generates polynomial parameters for each independent computation unit at each time step based on the input time-series data (historical context or local data). The shared basis matrix buffer module stores the shared basis matrix and several low-order powers corresponding to each independent computation unit in on-chip storage to avoid repeatedly reading the complete time-varying dense matrix from external storage at each time step. The coefficient domain operation module performs polynomial multiplication, convolution, modular reduction, or prefix reduction in the coefficient domain. The matrix block computation module represents the polynomial parameters as matrix blocks when needed, or directly performs multiplication, addition, projection, and fusion on the matrix blocks. The mode scheduling module switches the execution path between parallel and recursive views according to task requirements. The output fusion module completes the execution of the artificial intelligence model.

[0202] During the training phase of the manually executed model, the coefficient domain operation module first performs coefficient-level synthesis of the state transition operators; then, the matrix block computation module performs block matrix multiplication and addition operations at necessary locations. This reduces the repeated materialization of the complete time-varying dense matrix and lowers the read / write pressure on the HBM. For local block multiplications in query terms, key terms, and causal masks in the matrix parameters, a dedicated matrix computation unit can be invoked to improve the throughput at the matrix block level.

[0203] During the inference phase of the manually executed model, polynomial parameters, shared-level matrices, and some intermediate coefficients are stored in on-chip high-speed memory, with results written back to external memory only when necessary. Since time-series model operators can be constructed with polynomial parameters and shared-basis matrices, there is no need to explicitly load new, complete time-varying dense matrices in each practice unit, which helps reduce incremental inference latency.

[0204] When the manual execution model is applied to a multi-card cluster system, the target data to be synchronized is less, and the data can be transmitted across devices in a form that is closer to "control flow + a small amount of shared base", which can reduce the communication burden.

[0205] The matrix block computation unit supports mixed-precision execution, including but not limited to floating-point formats such as FP16, BF16, TF32, FP8, and INT8, or combinations thereof; while the coefficient domain operation module can use the same or different numerical precision as the matrix block computation module. By combining high-throughput matrix multiplication and addition with low-dimensional coefficient control, the effective computation density per unit time can be improved while ensuring that the main dynamic structure remains unchanged.

[0206] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0207] One embodiment of this application provides a workstation including a central processing unit and at least one graphics processor. The central processing unit is used to acquire input timing data and transmit the input timing data to the at least one graphics processor. The at least one graphics processor is used to execute the steps of the artificial intelligence model execution method provided in the above embodiment.

[0208] When the artificial intelligence model execution method is applied in a single-card scenario, the computing device can be a workstation, which includes an execution module (server or terminal). The execution module is equipped with a central processing unit (CPU) and at least one graphics processing unit (GPU). The CPU is used to acquire input timing data. The GPU acquires the input timing data from the CPU and executes the steps of the artificial intelligence model execution method provided in the above embodiments based on the input timing data.

[0209] This application provides a workstation that includes at least one graphics processor for executing the steps of the artificial intelligence model execution method provided in the above embodiments. The workstation then has all the beneficial effects of the artificial intelligence model execution method, which will not be elaborated here.

[0210] One embodiment of this application provides a multi-GPU cluster system, including a scheduling host and multiple computing nodes. Each computing node includes a central processing unit (CPU) and multiple graphics processing units (GPUs). The scheduling host is used to allocate execution tasks of an artificial intelligence model to the multiple computing nodes. The CPU in each computing node acquires input timing data and transmits the input timing data to the corresponding multiple GPUs, so that the GPUs execute the steps of the artificial intelligence model execution method provided in the above embodiment based on the execution tasks.

[0211] In the application of the artificial intelligence model execution method in a multi-GPU cluster scenario, the computing device can be a multi-GPU cluster system, which includes a scheduling host and multiple computing nodes. A computing node refers to a server or terminal. Each computing node includes a central processing unit (CPU) and multiple graphics processing units (GPUs). The scheduling host allocates execution tasks for the artificial intelligence model to the multiple computing nodes, transmitting input timing data to each node during task allocation. The CPU in each computing node acquires the input timing data and transmits it to the corresponding GPUs. Upon receiving the input timing data, the GPUs execute the artificial intelligence model execution method provided in the above embodiments based on the allocated execution tasks.

[0212] The graphics processor in the multi-GPU cluster system provided in this application embodiment is used to execute the steps of the artificial intelligence model execution method provided in the above embodiment. Therefore, the multi-GPU cluster system has all the beneficial effects of the artificial intelligence model execution method, which will not be repeated here.

[0213] In one embodiment, the graphics processor includes static random access memory, and the polynomial parameters and shared basis matrix are stored in the static random access memory.

[0214] Graphics processors include static random access memory (HRAM), which is shared memory, registers, and other on-chip storage. The polynomial parameters and shared basis matrices used in the above process can be stored in HRAM, which can reduce repeated reads and writes to HRAM and thus alleviate the memory wall problem.

[0215] In an optional embodiment, during the execution of the above method, many processes can be handled in the coefficient domain, and materialization / explicit instantiation can be delayed, reducing repeated reads and writes to the HBM and thus alleviating the memory wall problem. Coefficient domain processing can be implemented in the central processing unit (CPU) or a computational module for coefficient domain processing can be deployed in the graphics processing unit (GPU), allowing coefficient domain processing to be performed directly on the GPU.

[0216] Based on the same inventive concept, this application also provides an artificial intelligence model execution device for implementing the artificial intelligence model execution method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the artificial intelligence model execution device provided below can be found in the limitations of the artificial intelligence model execution method described above, and will not be repeated here.

[0217] In one embodiment, such as Figure 13 As shown, an artificial intelligence model execution device is provided, including: a determining module 11, a constructing module 12, and an execution module 13, wherein:

[0218] The determination module 11 is used to determine the polynomial parameters of each independent computational unit in the artificial intelligence model based on the acquired input time series data.

[0219] Module 12 is used to obtain the shared basis matrix of each independent computing unit and to construct a time series model operator for each independent computing unit based on the shared basis matrix and polynomial parameters.

[0220] Execution module 13 is used to execute an artificial intelligence model based on the input time series data and the time series model operators of all independent computing units.

[0221] In one embodiment, the determining module 11 includes an acquisition unit and a first determining unit. The acquisition unit is used to acquire input time-series data; the first determining unit is used to generate a network based on the input time-series data and a pre-trained coefficient generator to obtain the polynomial parameters of each independent computation unit; the polynomial coefficients of each independent computation unit include the polynomial parameters corresponding to each time step of the input time-series data.

[0222] In one embodiment, the construction module 12 is specifically used to construct a state transition operator based on a first polynomial parameter and a shared basis matrix; construct an input projection operator based on a second polynomial parameter and a shared basis matrix; and construct an output projection operator based on a third polynomial parameter and a shared basis matrix.

[0223] In one embodiment, the execution module 13 includes a generation unit and an execution unit. The generation unit is used to generate a preset view for each independent computing unit, based on task requirements and time series model operators; the preset view includes a parallel view or a recursive view; the task requirements include at least one of the following: the pattern of the artificial intelligence model, the sequence length of the input time series data, the real-time requirements for executing the artificial intelligence model, and the hardware conditions for executing the artificial intelligence model; the execution unit is used to execute the artificial intelligence model based on the input time series data and the preset views of all independent computing units.

[0224] In one embodiment, the determining module 11 is further configured to determine a recursive view based on the time series model operator and a preset recursive formula.

[0225] In one embodiment, the determining module 11 is further configured to convert the recursive view into a parallel view according to a preset state-space duality calculation formula.

[0226] In one embodiment, the execution unit is specifically used to determine a recursive view for each independent computing unit based on the time series model operator; substitute the input time series data into the recursive view to determine the first output result; and fuse the first output results of all independent computing units to obtain the inference result of the artificial intelligence model.

[0227] In one embodiment, the execution unit is further configured to, for each independent computing unit, determine a parallel view based on the time-series model operator; determine a second output result based on the input time-series data and the parallel view; fuse the second output results of all independent computing units to obtain a fusion result of the artificial intelligence model; and complete the training of the artificial intelligence model based on the fusion result.

[0228] In one embodiment, the execution unit is further configured to, for each independent computing unit, determine the parallel view of the previous time step based on the model operator corresponding to the previous time step; determine the output result of the current time step based on the input data of the current time step and the parallel view of the previous time step; fuse the output results of all independent computing units at the current time step to obtain the fusion result of the current time step, and calculate the local gradient based on the fusion result of the current time step; determine the model operator corresponding to the current time step based on the local gradient, the model operator corresponding to the previous time step, and the input data of the current time step, and use the model operator corresponding to the current time step as the model operator corresponding to the previous time step corresponding to the input data of the next time step, so as to determine the output result of the next time step.

[0229] In one embodiment, the execution unit is further configured to calculate the matrix parameters required for the parallel view of the previous time step according to the model operator corresponding to the current time step; the matrix parameters include query terms, key terms, and causal mask; calculate the causal mask in the coefficient domain, and determine the output result of the current time step based on the calculation result, the parallel view of the previous time step, query terms, key terms, and the input data of the current time step.

[0230] In one embodiment, the execution unit is further configured to perform synchronous communication between multiple execution modules on the first target data to obtain synchronized first target data; the first target data includes polynomial parameters, and the execution module includes multiple independent computing units; a time series model operator is obtained based on the synchronized first target data and the shared basis matrix; for each independent computing unit, a recursive view is determined based on the time series model operator; the input time series data is substituted into the recursive view to determine the third output result; and the third output results of all independent computing units are fused to obtain the inference result of the artificial intelligence model.

[0231] In one embodiment, the execution unit is further configured to calculate the local gradient for each execution module based on the fusion result; the execution module includes multiple independent computing units; perform synchronous communication between multiple execution modules on the second target data, and update the network parameters of the artificial intelligence model based on the synchronized second target data; the target data includes the local gradient, polynomial parameters, and shared basis matrix.

[0232] Each module in the aforementioned artificial intelligence model execution device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0233] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 14As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an artificial intelligence model execution method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0234] Those skilled in the art will understand that Figure 14 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0235] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0236] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0237] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0238] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0239] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0240] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for executing an artificial intelligence model, characterized in that, The method includes: Based on the acquired input time series data, determine the polynomial parameters of each independent computational unit in the artificial intelligence model; Obtain the shared basis matrix for each independent computing unit, and construct a time series model operator for each independent computing unit based on the shared basis matrix and the polynomial parameters; The artificial intelligence model is executed based on the input time series data and the time series model operators of all the independent computing units.

2. The method according to claim 1, characterized in that, The step of determining the polynomial parameters of each independent computational unit in the artificial intelligence model based on the acquired input time-series data includes: Obtain the input timing data; Based on the input time series data and the pre-trained coefficient generation network, the polynomial parameters of each independent computing unit are obtained; the polynomial coefficients of each independent computing unit include the polynomial parameters corresponding to each time step of the input time series data.

3. The method according to claim 1, characterized in that, The time-series model operator includes a state transition operator, an input projection operator, and an output projection operator. The polynomial parameters include a first polynomial parameter, a second polynomial parameter, and a third polynomial parameter. The step of constructing a time-series model operator for each independent computational unit based on the shared basis matrix and the polynomial parameters includes: The state transition operator is constructed based on the first polynomial parameters and the shared basis matrix; The input projection operator is constructed based on the second polynomial parameters and the shared basis matrix; The output projection operator is constructed based on the third polynomial parameters and the shared basis matrix.

4. The method according to claim 1, characterized in that, The step of executing an artificial intelligence model based on the input time-series data and the time-series model operators of all the independent computing units includes: For each independent computing unit, a preset view is generated based on task requirements and the time series model operator; the preset view includes a parallel view or a recursive view; the task requirements include at least one of the following: the pattern of the artificial intelligence model, the sequence length of the input time series data, the real-time requirements for executing the artificial intelligence model, and the hardware conditions for executing the artificial intelligence model; An artificial intelligence model is executed based on the input time-series data and the preset view of all the independent computing units.

5. The method according to claim 4, characterized in that, The method for determining the recursive view includes: The recursive view is determined based on the time series model operator and the preset recursive formula.

6. The method according to claim 5, characterized in that, The method for determining the parallel views includes: The recursive view is converted into the parallel view according to the preset state-space duality calculation formula.

7. The method according to claim 4, characterized in that, When the mode of the artificial intelligence model is in the inference phase, the artificial intelligence model is executed based on the input time-series data and the time-series model operators of all the independent computing units, including: For each of the independent computing units, the recursive view is determined based on the time-series model operator; Substitute the input timing data into the recursive view to determine the first output result; The first output results of all the independent computing units are fused to obtain the inference result of the artificial intelligence model.

8. The method according to claim 4, characterized in that, When the mode of the artificial intelligence model is in the training phase, executing the artificial intelligence model based on the input time-series data and the time-series model operators of all the independent computing units includes: For each of the independent computing units, the parallel view is determined based on the time-series model operator; The second output result is determined based on the input timing data and the parallel view; The second output results of all the independent computing units are fused to obtain the fusion result of the artificial intelligence model, and the training of the artificial intelligence model is completed based on the fusion result.

9. The method according to claim 4, characterized in that, The input time-series data includes the input data at the current time and the input data at the next time. The time-series model operators include the model operators corresponding to the previous time and the model operators corresponding to the current time. The step of executing the artificial intelligence model based on the input time-series data and the time-series model operators of all the independent computing units includes: For each independent computing unit, the parallel view of the previous time step is determined based on the model operator corresponding to the previous time step. The output result at the current moment is determined based on the input data at the current moment and the parallel view at the previous moment; The output results of all the independent computing units at the current time are fused to obtain the fused result at the current time, and the local gradient is calculated based on the fused result at the current time. Based on the local gradient, the model operator corresponding to the previous time step, and the input data at the current time step, the model operator corresponding to the current time step is determined, and the model operator corresponding to the current time step is used as the model operator corresponding to the previous time step for the input data at the next time step, so as to determine the output result at the next time step.

10. The method according to claim 9, characterized in that, Determining the output result at the current moment based on the input data at the current moment and the parallel view at the previous moment includes: The matrix parameters required to calculate the parallel view of the previous time step are calculated based on the model operator corresponding to the current time step; the matrix parameters include query terms, key terms, and causal masks; The causal mask is calculated in the coefficient domain, and the output result at the current moment is determined based on the calculation result, the parallel view of the previous moment, the query item, the key item, and the input data at the current moment.

11. The method according to claim 7, characterized in that, In the case where the mode of the artificial intelligence model is the inference phase of a multi-GPU cluster, the execution of the artificial intelligence model based on the input time-series data and the time-series model operators of all the independent computing units includes: Synchronous communication is performed between multiple execution modules to obtain synchronized first target data; the first target data includes polynomial parameters, and the execution module includes multiple independent computing units. The time series model operator is obtained based on the synchronized first target data and the shared basis matrix; For each independent computational unit, the recursive view is determined based on the time-series model operator; Substitute the input timing data into the recursive view to determine the third output result; The third output results of all the independent computing units are fused to obtain the inference result of the artificial intelligence model.

12. The method according to claim 8, characterized in that, In the case where the training phase of the artificial intelligence model is a multi-GPU cluster, the step of training the artificial intelligence model based on the fusion result includes: For each execution module, a local gradient is calculated based on the fusion result; the execution module includes multiple independent computing units. The second target data is used for synchronous communication among multiple execution modules to update the network parameters of the artificial intelligence model based on the synchronized second target data; the second target data includes the local gradient, the polynomial parameters, and the shared basis matrix.

13. A workstation, characterized in that, The method includes a central processing unit and at least one graphics processor, wherein the central processing unit is configured to acquire input timing data and transmit the input timing data to the at least one graphics processor, and the at least one graphics processor is configured to perform the steps of the method as described in any one of claims 1 to 12.

14. A multi-card cluster system, characterized in that, It includes a scheduling host and multiple computing nodes, each computing node including a central processing unit and multiple graphics processing units; The scheduling host is used to allocate execution tasks of artificial intelligence models to the plurality of computing nodes; the central processing unit in each computing node acquires input timing data and transmits the input timing data to the corresponding plurality of graphics processing units, so that the graphics processing unit performs the steps of the method as described in any one of claims 1 to 12 based on the execution task.

Images