Methods based on tensor structure optimization artificial intelligence models

Abstract

This invention provides a method for optimizing an artificial intelligence model based on tensor structures. The method includes: replacing the linear mapping weight matrix in an initial artificial intelligence model with a tensor structure to obtain a target artificial intelligence model. The tensor structure includes a target tensor and / or a target tensor network. The target artificial intelligence model is used to process a target task, which includes at least one of natural language processing, logical and mathematical reasoning, code programming, and multimodal content generation. This invention achieves structured compression of the artificial intelligence model through target tensor and / or target tensor network technology, significantly reducing the number of model parameters, disk space usage, and GPU memory usage while effectively reducing computational complexity and significantly improving the inference and training speed of the model.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and tensor technology, and in particular to a method for optimizing artificial intelligence models based on tensor structures. Background Technology

[0002] Large Language Models (LMs) based on the Transformer architecture are among the most watched technologies in the field of artificial intelligence. The performance improvement of large language models follows the "Scaling Law", which states that the model performance is usually positively correlated with the number of parameters, computational cost, and training data volume, and will exhibit "emergent abilities" after reaching a certain scale.

[0003] However, this pursuit of extreme scale brings significant resource challenges. Taking cutting-edge models such as the fourth-generation generative pre-trained transformer 4 (GPT-4) as examples, their high training costs and massive inference energy consumption reveal the resource bottleneck of focusing primarily on increasing model size. This bottleneck severely limits the deployment and application of large models in edge computing and low-computing-power scenarios. Therefore, an effective solution is urgently needed to address these issues, thereby reducing the training and usage costs of artificial intelligence models. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a method for optimizing artificial intelligence models based on tensor structures.

[0005] This invention provides a method for optimizing artificial intelligence models based on tensor structures, comprising:

[0006] A target artificial intelligence model is obtained by replacing the linear mapping weight matrix in the initial artificial intelligence model with a tensor structure. The tensor structure includes a target tensor and / or a target tensor network. The target artificial intelligence model is used to process a target task, which includes at least one of natural language processing, logical and mathematical reasoning, code programming, and multimodal content generation.

[0007] According to a method for optimizing an artificial intelligence model based on a tensor structure provided by the present invention, when the tensor structure is the target tensor, the method involves replacing the linear mapping weight matrix in the initial artificial intelligence model with the tensor structure to obtain the target artificial intelligence model, comprising:

[0008] Based on the number of parameters of the feature dimensions involved in the operation in the linear mapping weight matrix, the target tensor is constructed, and the target tensor includes at least two local gate tensors in parallel.

[0009] Based on the target tensor, the linear mapping weight matrix in the initial artificial intelligence model is replaced to obtain the initialized artificial intelligence model to be trained;

[0010] Based on the task sample set, the artificial intelligence model to be trained is restored and trained to obtain the target artificial intelligence model. The task sample set includes at least one of the following: natural language processing sample set, logic and mathematical reasoning sample set, code programming sample set, and multimodal content generation sample set.

[0011] According to a method for optimizing an artificial intelligence model based on tensor structure provided by the present invention, the feature dimensions involved in the operation of the linear mapping weight matrix include input feature dimensions and output feature dimensions.

[0012] The construction of the target tensor based on the number of parameters of the feature dimensions involved in the operation in the linear mapping weight matrix includes:

[0013] Determine the dimensions of physical indicators;

[0014] Based on the physical index dimension, determine the number of physical indices corresponding to the input feature dimension and the number of physical indices corresponding to the output feature dimension;

[0015] Based on the number of physical indicators corresponding to the input feature dimension, the number of physical indicators corresponding to the output feature dimension, and the target compression ratio, the number of local gate tensors N and the number of input and output physical indicators of the local gate tensor are determined with the physical indicator dimension as the input physical indicator dimension and the output physical indicator dimension of the local gate tensor. The number of input physical indicators is less than the number of physical indicators corresponding to the input feature dimension, and the number of output physical indicators is less than the number of physical indicators corresponding to the output feature dimension.

[0016] The target tensor is obtained by concatenating the N local gate tensors according to a parallel summation architecture.

[0017] According to a method for optimizing an artificial intelligence model based on tensor structures provided by the present invention, after replacing the linear mapping weight matrix in the initial artificial intelligence model with a tensor structure to obtain the target artificial intelligence model, the method further includes:

[0018] The input layer of the target artificial intelligence model encodes the target task to obtain the input vector corresponding to the target task.

[0019] Using the target tensor, the input vector is reshaped based on the physical index dimension to obtain a higher-order tensor corresponding to the number of physical indices corresponding to the input feature dimension;

[0020] By performing tensor contraction operations on the higher-order tensors using each of the local gate tensors, the branch output tensors of each of the local gate tensors are obtained;

[0021] The output tensors of each branch are summed and normalized according to the number of local gate tensors to obtain the target output tensor.

[0022] The target output tensor is decoded based on the output layer of the target artificial intelligence model to obtain the processing result of the target task.

[0023] According to a method for optimizing an artificial intelligence model based on tensor structure provided by the present invention, determining the number of physical indicators corresponding to the input feature dimension and the number of physical indicators corresponding to the output feature dimension based on the physical indicator dimension includes:

[0024] For any one of the input feature dimensions and the output feature dimensions, the dimension is transformed according to the following formula to obtain the number of physical indicators corresponding to the dimension:

[0025] D=d L

[0026] Where D is the dimension, d is the physical index dimension, and L is the number of physical indices corresponding to the dimension.

[0027] According to a method for optimizing an artificial intelligence model based on tensor structure provided by the present invention, the process of determining the number of input physical indices and the number of output physical indices includes:

[0028] The number of input physical indices and the number of output physical indices of the local gate tensor are calculated according to the following formula:

[0029] N = L1 - L2 + 1

[0030] Where N is the number of local gate tensors, L1 is the number of physical indices corresponding to the input feature dimension and L2 is the number of input physical indices of the local gate tensor, or L1 is the number of physical indices corresponding to the output feature dimension and L2 is the number of output physical indices of the local gate tensor.

[0031] According to a method for optimizing an artificial intelligence model based on tensor structure provided by the present invention, the step of concatenating N local gate tensors according to a parallel summation computational architecture to obtain the target tensor includes:

[0032] The N local gate tensors are concatenated using a parallel summation architecture to obtain the initial tensor;

[0033] The backpropagation algorithm in machine learning is used to initialize the structural parameters of the initial tensor to obtain the target tensor.

[0034] According to a method for optimizing an artificial intelligence model based on tensor structure provided by the present invention, the method employs the backpropagation algorithm in machine learning to initialize the structural parameters of the initial tensor to obtain the target tensor, comprising:

[0035] Obtain the identity matrix corresponding to the linear mapping weight matrix;

[0036] Input the identity matrix into the initial tensor and output the equivalent weight matrix corresponding to the initial tensor;

[0037] Based on the linear mapping weight matrix and the equivalent weight matrix, the loss value is calculated using the Frobenius norm as the loss function.

[0038] Based on the loss value, the gate tensor parameters of the initial tensor are adjusted using the gradient descent method;

[0039] Continue initializing the gate tensor parameters of the adjusted initial tensor until the loss value converges, the initialization is complete, and the target tensor is obtained.

[0040] According to a method for optimizing an artificial intelligence model based on a tensor structure provided by the present invention, when the tensor structure is the target tensor network, the method involves replacing the linear mapping weight matrix in the initial artificial intelligence model with the tensor structure to obtain the target artificial intelligence model, comprising:

[0041] Based on the number of parameters of the feature dimensions involved in the operation in the linear mapping weight matrix, at least one target tensor is constructed, the target tensor including at least two local gate tensors in parallel;

[0042] The target tensors are layered and connected to obtain the target tensor network;

[0043] Based on the target tensor network, the linear mapping weight matrix in the initial artificial intelligence model is replaced to obtain the initialized artificial intelligence model to be trained;

[0044] Based on the task sample set, the artificial intelligence model to be trained is restored and trained to obtain the target artificial intelligence model. The task sample set includes at least one of the following: natural language processing sample set, logic and mathematical reasoning sample set, code programming sample set, and multimodal content generation sample set.

[0045] According to the present invention, a method for optimizing an artificial intelligence model based on tensor structure is provided, wherein the step of performing recovery training on the artificial intelligence model to be trained based on a task sample set to obtain the target artificial intelligence model includes:

[0046] Obtain the task sample set;

[0047] Based on the task sample set and the loss function, the artificial intelligence model to be trained is restored and trained to obtain the target artificial intelligence model. The loss function is the perplexity loss or cross-entropy loss of the prediction task.

[0048] The present invention also provides an apparatus for optimizing an artificial intelligence model based on tensor structure, comprising:

[0049] The replacement module is configured to use a tensor structure to replace the linear mapping weight matrix in the initial artificial intelligence model to obtain a target artificial intelligence model. The tensor structure includes a target tensor and / or a target tensor network. The target artificial intelligence model is used to process a target task, which includes at least one of natural language processing, logic and mathematical reasoning, code programming, and multimodal content generation.

[0050] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for optimizing an artificial intelligence model based on tensor structure as described above.

[0051] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for optimizing an artificial intelligence model based on tensor structures as described above.

[0052] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the method for optimizing an artificial intelligence model based on tensor structure as described above.

[0053] This invention provides a method for optimizing artificial intelligence (AI) models based on tensor structures. By replacing the linear mapping weight matrix in the initial AI model with a tensor structure, a target AI model is obtained. The tensor structure includes a target tensor and / or a target tensor network. The target AI model is used to process a target task, which includes at least one of natural language processing, logical and mathematical reasoning, code programming, and multimodal content generation. This invention achieves structured compression of AI models based on target tensors and / or target tensor networks, i.e., replacing the linear mapping weight matrix in the AI ​​model with a deep tensor structure. This significantly reduces the number of model parameters, disk space usage, and GPU memory usage, while effectively reducing computational complexity and significantly improving the inference and training speed of the model. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating the method for optimizing artificial intelligence models based on tensor structures provided by the present invention.

[0056] Figure 2 This is a flowchart illustrating the target tensor compression attention mechanism provided by the present invention.

[0057] Figure 3 This is one of the schematic diagrams illustrating the effect of controlling the compression ratio by adjusting the size of the local gate tensor provided by the present invention.

[0058] Figure 4 This is the second schematic diagram illustrating the effect of controlling the compression ratio by adjusting the size of the local gate tensor provided by the present invention.

[0059] Figure 5 This is a comparison chart showing the effects of the attention mechanism, matrix multiplication operator, and tensor structure implemented by the present invention on matrix multiplication.

[0060] Figure 6 This is a comparison chart of the training efficiency of different model configurations provided by this invention in a scenario of full parameter fine-tuning.

[0061] Figure 7 This is a schematic diagram of the device for optimizing artificial intelligence models based on tensor structures provided by the present invention.

[0062] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0064] First, a brief description of the relevant content involved in this invention will be given.

[0065] Large language models are essentially deep neural networks trained on massive amounts of data, possessing the ability to understand and generate natural language. The Bidirectional Encoder Representations from Transformers (BERT) model validated the effectiveness of the Transformer architecture. Subsequently, models such as the Chat Generative Pre-trained Transformer (ChatGPT) and the Large Language Model Meta AI (LLaMA) series, with their powerful human-computer interaction and reasoning capabilities, have driven the rapid development of generative artificial intelligence technology. Recently, some AI software has demonstrated outstanding performance in complex logic processing tasks by introducing "deep thinking" reasoning strategies, sparking in-depth industry attention to model reasoning capabilities. Currently, large language models are widely used in text processing, code generation, and scientific computing, and represent a core direction of current AI research.

[0066] The high training cost and massive inference energy consumption of large language models severely limit their deployment and application in edge computing and low computing power scenarios.

[0067] Meanwhile, multiple studies have shown significant parameter redundancy within large models. For example, the "Lottery Ticket Hypothesis" suggests that dense neural networks may contain smaller subnetworks that, after independent training, achieve performance comparable to the original network. Some studies indicate that many attention heads in the Transformer architecture can be removed without significantly impacting model performance. These findings suggest that the weight matrices in deep networks often exhibit low-rank characteristics, offering substantial room for compression. Therefore, developing efficient model compression techniques holds promise for significantly reducing operational costs while maintaining the core capabilities of the model.

[0068] Currently, in the field of large-scale AI model compression, matrix multiplication operators are used to compress the parameters of large models. As a classic tensor network model, the core idea of ​​matrix multiplication operators is to decompose the massive weight matrix in the model into a series of linear chained products of low-order tensors. The specific implementation process of this method includes the following steps:

[0069] 1. Matrix Reshaping: Select the two-dimensional weight matrix W in the linear layer of the large model. To adapt to the tensor network structure, the input D of the weight matrix is ​​reshaped. in / outputD out The dimensions are split into several sub-dimensions and multiplied to form a product relationship. This allows the two indices of the original matrix to be recoded into multi-dimensional indices, achieving a reshaping mapping from a two-dimensional matrix to a higher-order tensor. When D... in Or D out When a product cannot be precisely represented as a product of preset physical dimensions, it can be made to satisfy the above product relationship by using zero padding, dimension alignment, or selecting a non-uniform set of physical dimensions.

[0070] 2. Tensor Decomposition: For the reshaped higher-order tensor, the algorithm sequentially performs Singular Value Decomposition (SVD) along a specific dimension. Each decomposition operation separates a lower-order core tensor from the original tensor and generates virtual indices connecting to the next core tensor. This process continues until the entire higher-order tensor is completely decomposed into a linear chain structure consisting of multiple core tensors connected sequentially, i.e., matrix multiplication operator form.

[0071] 3. Truncation Compression: In order to achieve data compression during the decomposition process, the algorithm introduces a truncation mechanism: only the largest value is retained. We take 1 singular value and its corresponding vector, and discard the remaining smaller singular values. Here... Known as the "bond dimension," this is a key hyperparameter whose value directly determines the degree of information retention. A smaller bond dimension means a higher compression ratio, but it can lead to a loss of model accuracy.

[0072] 4. Recovery Training: The truncation operation described above usually leads to a decrease in model performance. Therefore, end-to-end fine-tuning is required. In this process, the topology of the matrix multiplication operator is kept unchanged, and backpropagation is performed using a small amount of data to update the parameters of the matrix multiplication operator to compensate for the accuracy loss caused by truncation, so that the model performance is close to the original level.

[0073] Although compression methods based on matrix multiplication operators can theoretically significantly reduce the number of model parameters, in the practical engineering deployment and application of large language models, this method still faces the following two significant drawbacks:

[0074] 1. Initialization and construction are difficult, and the high key dimension leads to the loss of compression advantages.

[0075] The initialization of the matrix product operator structure relies on global singular value decomposition (SVD) of the original weight matrix. This algorithm requires loading the entire huge dense matrix into video memory at once for high-dimensional tensor rearrangement and decomposition. For the ultra-large-scale weight matrices commonly found in large language models, this process places extremely high demands on video memory, often exceeding the video memory limit of a single graphics card, thus making it impossible to complete parameter initialization.

[0076] More importantly, the advantage of matrix multiplication operators in quantum physics is based on the "area law," meaning they can only efficiently represent systems with low entanglement entropy. However, while the weight matrices of large language models have parameter redundancy, their internal characteristics often exhibit complex global long-range correlations, which do not conform to the local low-entanglement structural characteristics that matrix multiplication operators excel at. To maintain model accuracy, it is usually necessary to incorporate the bond dimension. Set at a higher level (e.g.) ≈200. This requirement for high key dimension makes the matrix multiplication operator model no longer sparse in mathematical structure, which not only offsets the storage advantage brought by parameter compression, but also causes the subsequent computational workload to increase quadratically.

[0077] 2. The reasoning computation is highly complex, and the dynamic memory explodes.

[0078] The chained structure of matrix multiplication operators dictates a high degree of sequential dependency in their computation; input data must be processed sequentially with the core tensors on the chain. This serial mechanism cannot leverage the massively parallel computing advantages of modern Graphics Processing Units (GPUs), leading to a significant increase in inference latency. Furthermore, when processing batch data, the high-dimensional tensor shrinkage process of matrix multiplication operators (especially in high-key dimensions) presents challenges. The following (below) generates intermediate state tensors with enormous dimensions. Experiments show that these temporarily generated intermediate variables not only significantly increase floating-point operations (FLOPs) but also cause runtime dynamic memory usage to far exceed that of the uncompressed original attention mechanism. This "intermediate state explosion" phenomenon makes the model compressed by the matrix multiplication operator difficult to deploy on memory-constrained devices, thus violating the original intention of model compression.

[0079] Tensor Networks (TNNs) are a set of mathematical tools originating from quantum physics used to solve the "curse of dimensionality" problem in high-dimensional quantum systems. In quantum many-body physics, the Hilbert space dimension of the system's wavefunction grows exponentially with the number of particles. Tensor Networks leverage the property that quantum states typically satisfy the "area law" of entanglement entropy, retaining only physically important low-entanglement states. This allows for efficient characterization and computation of high-dimensional quantum states with a parameter scale far smaller than the full space dimension.

[0080] Matrix Product States (MPS) are the most typical structure, decomposing high-order tensors into one-dimensional chain-like lower-order tensors. By truncating singular values ​​to control the "bond dimension" between the chains, it significantly reduces parameter complexity while maintaining approximation accuracy. The Density Matrix Renormalization Group (DMRG) algorithm proves that MPS can approximate the ground state of a one-dimensional quantum system with high accuracy by truncating small singular values. Further research shows that MPS can effectively simulate quantum computing processes with low entanglement. This method can significantly reduce parameter complexity from exponential to polynomial level. The above theoretical basis illustrates that, through appropriate tensor decomposition structures, high-dimensional data can be represented while preserving key information correlations. It also provides a theoretical basis for using tensor decomposition techniques to compress high-dimensional weight matrices in neural networks. By representing high-dimensional weight tensors in the model as tensor networks with specific structures and truncating or constraining the internal bond dimensions, the number of parameters and computational cost can be significantly reduced while maintaining the original model's ability to express the dependencies of input features as much as possible.

[0081] To address the resource bottlenecks caused by the continuous expansion of large language models, such as high storage costs, large GPU memory consumption during inference and training, and excessive computation time due to their massive parameter count, this invention utilizes the efficient structured representation provided by tensor structures to compress and accelerate the attention mechanism in artificial intelligence models, taking advantage of the significant parameter redundancy inherent in deep networks of large models.

[0082] The following is combined with Figures 1 to 8 The present invention describes a method for optimizing artificial intelligence models based on tensor structures.

[0083] Figure 1 This is a flowchart illustrating the method for optimizing artificial intelligence models based on tensor structures provided by the present invention, as shown below. Figure 1 As shown, the method includes the following:

[0084] Step 101: Using a tensor structure, replace the linear mapping weight matrix in the initial artificial intelligence model to obtain the target artificial intelligence model. The tensor structure includes a target tensor and / or a target tensor network. The target artificial intelligence model is used to process a target task. The target task includes at least one of natural language processing, logic and mathematical reasoning, code programming, and multimodal content generation.

[0085] First, the execution subject of the method for optimizing artificial intelligence models based on tensor structures provided by this invention can be a terminal, controller, or application program with task processing functions.

[0086] Specifically, the artificial intelligence model can be a large artificial intelligence model, such as a large language model; the target task includes at least one of natural language processing, logical and mathematical reasoning, code programming and multimodal content generation. Natural language tasks refer to tasks that use text to describe and process text, such as language processing tasks such as answering tasks, writing tasks or dialogue tasks.

[0087] The initial AI model can be a regular AI model, where the linear mapping weight matrix is ​​the linear mapping weight matrix within the attention mechanism of the Transformer architecture in the initial AI model.

[0088] Specifically, initial artificial intelligence models exhibit significant parameter redundancy. This invention addresses this by using an automatically differentiable tensor structure, specifically a quantum-inspired tensor structure, to replace the linear mapping weight matrix in the Transformer architecture. The tensor structure includes a target tensor and / or a target tensor network. The target tensor comprises at least two parallel local gate tensors, and the target tensor network comprises at least two hierarchically connected target tensors. Therefore, it can be understood as reconstructing the high-dimensional linear mapping weight matrix operations in existing artificial intelligence models into a parallel summation of multiple local gate tensors, thereby obtaining the target artificial intelligence model, which is then deployed in the execution entity.

[0089] Based on this, if a target task is received, the executing entity can call the target artificial intelligence model to process the target task and obtain the corresponding processing result.

[0090] The present invention provides a method for optimizing artificial intelligence models based on tensor structures. This method uses a target tensor and / or a target tensor network to replace the linear mapping weight matrix in the Transformer architecture of the artificial intelligence model. It reconstructs the high-dimensional linear mapping weight matrix operations into a parallel summation of multiple local gate tensors, leveraging the independence of these local gate tensors to achieve parallelization of the computation process. This structure, while eliminating redundant parameters and significantly compressing the model size, effectively shortens inference latency by reducing floating-point operations and dynamic memory usage, thereby significantly accelerating model inference on general-purpose hardware and improving task processing efficiency.

[0091] In one or more optional embodiments of the present invention, when the tensor structure is the target tensor, the step of replacing the linear mapping weight matrix in the initial artificial intelligence model with the tensor structure to obtain the target artificial intelligence model includes:

[0092] Based on the number of parameters of the feature dimensions involved in the operation in the linear mapping weight matrix, the target tensor is constructed, and the target tensor includes at least two local gate tensors in parallel.

[0093] Based on the target tensor, the linear mapping weight matrix in the initial artificial intelligence model is replaced to obtain the initialized artificial intelligence model to be trained;

[0094] Based on the task sample set, the artificial intelligence model to be trained is restored and trained to obtain the target artificial intelligence model. The task sample set includes at least one of the following: natural language processing sample set, logic and mathematical reasoning sample set, code programming sample set, and multimodal content generation sample set.

[0095] First, to clearly explain the computational logic of high-dimensional data in the target tensor, we will combine... Figure 2 A graphical representation of the target tensor is explained. See also Figure 2 , Figure 2 This is a flowchart illustrating the target tensor compression attention mechanism provided by this invention: In the graphical representation system, nodes represent tensors, i.e., containers for data; the line segments connecting the nodes represent the tensor's indices (dimensions). For example... Figure 2 As shown, the input data of an artificial intelligence model typically includes dataset-related dimensions such as batch size and sequence length, as well as hidden layer dimensions (Hidden Dim, 4096 in this example), i.e. feature dimensions. Figure 2In the graphical representation, dimensions unrelated to specific computational transformations (such as Batch and Seq Len) are represented by ellipses and red lines, while feature dimensions involved in the computation are represented by blue parts. Specifically, during the attention mechanism inference process, feature dimensions involved in the computation are represented by blue circles and blue lines connecting them, as well as green circles and green lines connecting them.

[0096] Correspondingly, the 4096×4096 weight matrix in the attention mechanism is represented as a green node with two solid green "legs," corresponding to the input and output feature dimensions, respectively. When the input data is multiplied by this matrix, graphically, the feature index line of the input data is connected to the input index line of the matrix, outputting a new blue tensor containing the red omitted dimension and the 4096 feature dimension.

[0097] The core of this embodiment lies in constructing a target tensor to replace the above-mentioned attention mechanism reasoning process.

[0098] In practical applications, based on the number of parameters in the feature dimensions involved in the computation of the linear mapping weight matrix, at least two local gate tensors matching the number of parameters can be selected or constructed to form the target tensor, such as... Figure 2 In the target tensor inference process, a green rectangle and the green lines connecting it represent a local gate tensor. Figure 2 It contains two local gate tensors that are in parallel.

[0099] Based on the target tensor, the linear mapping weight matrix of the initial artificial intelligence model is replaced with the target tensor to obtain the initialized artificial intelligence model to be trained. Furthermore, the artificial intelligence model to be trained is subjected to recovery training in at least one of the following capabilities: natural language processing, logical and mathematical reasoning, code programming, and multimodal content generation. That is, the artificial intelligence model to be trained is recovered and trained based on the task sample set to obtain the target artificial intelligence model.

[0100] In this embodiment of the invention, a target tensor is formed by at least two parallel local gate tensors to replace the linear mapping weight matrix, thereby achieving structured compression of the artificial intelligence model. This significantly reduces the number of model parameters and memory usage, while also reducing computational complexity and improving inference and training speed.

[0101] Optionally, the feature dimensions involved in the operation in the linear mapping weight matrix include input feature dimensions and output feature dimensions;

[0102] The construction of the target tensor based on the number of parameters of the feature dimensions involved in the operation in the linear mapping weight matrix includes:

[0103] Determine the dimensions of physical indicators;

[0104] Based on the physical index dimension, determine the number of physical indices corresponding to the input feature dimension and the number of physical indices corresponding to the output feature dimension;

[0105] Based on the number of physical indicators corresponding to the input feature dimension, the number of physical indicators corresponding to the output feature dimension, and the target compression ratio, the number of local gate tensors N and the number of input and output physical indicators of the local gate tensor are determined using the physical indicator dimension as the input physical indicator dimension and the output physical indicator dimension of the local gate tensor. The number of input physical indicators is less than the number of physical indicators corresponding to the input feature dimension, and the number of output physical indicators is less than the number of physical indicators corresponding to the output feature dimension.

[0106] The target tensor is obtained by concatenating the N local gate tensors according to a parallel summation architecture.

[0107] Specifically, the number of input physical indices represents the number of physical indices at the input end of the local gate tensor; the number of output physical indices represents the number of physical indices at the output end of the local gate tensor.

[0108] In practical applications, the target tensor involves reshaping the input feature dimensions. Specifically, assuming the feature dimension (hidden layer dimension) of the original input data x is D, this embodiment sets a physical index dimension d, and reshapes the feature dimension D (input feature dimension D) into a target tensor. in Or output feature dimension D out This can be expressed as the product of L d's (i.e., satisfying the relation D=d). L When the input feature dimension D in Or output feature dimension D out When a product cannot be precisely represented as a product of preset physical dimensions, it can be made to satisfy the above product relationship by using zero padding, dimension alignment, or selecting a non-uniform set of physical dimensions.

[0109] Under the above definition, the original input data x is reshaped into a high-order tensor with L physical indices, each indice having a dimension of d. The general mathematical expression for this transformation process is as follows:

[0110]

[0111] in, Let be the set of real numbers, D be the feature dimension of data x, d be the physical index dimension, and L represent the quantity of d, i.e., the number of physical indices. Furthermore, in the above formula, the ellipsis "..." corresponding to data x on the left and the higher-order tensor on the right... The first ellipsis "..." in each case represents a dimension such as batch size that is unrelated to the operation.

[0112] exist Figure 2 In the example shown, the physical index dimension d=2 is selected as an example. Therefore, for input data x with a feature dimension of 4096 (shape […,4096]), its feature dimension needs to be represented as a product of 12 twos (i.e., 2^2 ... 12 =4096). During inference, the omitted dimensions remain unchanged, and the original input is reshaped into a high-order tensor χ with 12 physical indicators, each with a dimension of 2, i.e., input reshaping. For example, in the inference process of the attention mechanism, the feature dimensions involved in the operation are represented by blue rectangles and blue lines connected to the blue rectangles, and green rectangles and green lines connected to the green rectangles.

[0113] The number of local gate tensors can be calculated based on the target compression ratio and physical performance metrics, using the formula for determining the number of local gate tensors. The formula for determining the number of local gate tensors is:

[0114] N * d 2-2N =Compression ratio, N>0

[0115] Where d is the physical index dimension and N is the number of local gate tensors.

[0116] It should be noted that N calculated based on the formula for determining the number of local gate tensors may not be an integer. In general, N can be rounded to obtain the final number of local gate tensors. Alternatively, depending on the actual needs, either rounding or rounding up can be used to process N to obtain the final number of local gate tensors.

[0117] After obtaining the input feature dimension D in and output feature dimension D out Based on the corresponding number of physical indicators, the physical indicator dimension can be used as the input physical indicator dimension and output physical indicator dimension of the local gate tensor, and then based on the input feature dimension D... in Given the target compression ratio, determine the number of input physical indices for the local gate tensor, and based on the output feature dimension D... out And the target compression ratio, determine the number of output physical indices of the local gate tensor, or, when the input feature dimension D in Or output feature dimension D out When a product of physical dimensions cannot be precisely represented as a preset product, zero-padding, dimension alignment, or selection of a non-uniform set of physical dimensions can be used to satisfy the aforementioned product relationship. Furthermore, to ensure compression efficiency and reduce data processing volume, the number of input physical metrics must be less than the number of physical metrics corresponding to the input feature dimensions, and the number of output physical metrics must be less than the number of physical metrics corresponding to the output feature dimensions.

[0118] Specifically, for any one of the input feature dimensions and the output feature dimensions, the dimension is transformed according to the following formula to obtain the number of physical indicators corresponding to the dimension:

[0119] D=d L

[0120] Where D is the dimension, d is the physical index dimension of the local gate tensor, and L is the number of physical indices corresponding to the dimension.

[0121] After obtaining the number N local gate tensors, these N local gate tensors are concatenated using a parallel summation architecture to obtain the target tensor. This structure overcomes the limitation of traditional target tensor networks, which must perform sequential, incremental shrinking computations. It breaks down the originally massive overall computation into multiple independent and concurrent local tensor operations. This highly parallel structure effectively utilizes the massively parallel computing resources of GPUs, improving inference speed. Simultaneously, this structure reduces floating-point operations and runtime memory usage, achieving comprehensive compression of the model in terms of parameter size, computational complexity, and inference time. Furthermore, this parallel structure avoids the training difficulties caused by excessively deep network layers, allowing the model to directly utilize the backpropagation algorithm for stable parameter initialization, no longer limited by the resource bottleneck caused by global decomposition of extremely large matrices.

[0122] Optionally, after replacing the linear mapping weight matrix in the initial artificial intelligence model with a tensor structure to obtain the target artificial intelligence model, the method further includes:

[0123] The target task is encoded based on the input layer of the target artificial intelligence model to obtain the input vector corresponding to the target task;

[0124] Using the target tensor, the input vector is reshaped based on the physical index dimension to obtain a higher-order tensor corresponding to the number of physical indices corresponding to the input feature dimension;

[0125] By performing tensor contraction operations on the higher-order tensors using each of the local gate tensors, the branch output tensors of each of the local gate tensors are obtained;

[0126] The output tensors of each branch are summed and normalized according to the number of local gate tensors to obtain the target output tensor.

[0127] The target output tensor is decoded based on the output layer of the target artificial intelligence model to obtain the processing result of the target task.

[0128] In practical applications, feature extraction, i.e., encoding, can be performed on the target task to obtain the input vector, and then processed according to D=d L The setting reshapes the input vector to obtain the reshaped higher-order input tensor, i.e., the higher-order tensor.

[0129] Furthermore, an operational architecture based on parallel summation of local gate tensors is proposed. For example... Figure 2 As shown in the "Target Tensor Inference Process" on the right, the computation can be performed in multiple parallel branches. Figure 2 The example in the figure shows two branches operating simultaneously. During this process, the input tensor maintains its reshaped physical index structure (as shown by the blue rectangle at the bottom of the figure). By configuring different local gate tensors (green rectangles at the top of the figure) to connect to different combinations of physical indices, the entire input information can be traversed and processed. Each parallel branch independently performs tensor shrinkage operations on the same input tensor, obtaining branch output tensors respectively. Then, a normalization operator is applied to all branch outputs, i.e., the branch outputs are summed and normalized according to the number of branches, resulting in the final output tensor, i.e., the target output tensor. Further, text recovery processing, i.e., decoding, is performed on the target output tensor to obtain the processing result of the target task.

[0130] The formula for summation and normalization based on the number of branches is as follows:

[0131]

[0132] Where Y is the target output tensor, N is the number of branches, i.e., the number of local gate tensors, and n represents the nth branch. (n) This represents the branch output tensor of the nth local gate tensor.

[0133] The target tensor significantly reduces the intermediate tensor dimension (tensor dimension in the inference process) during computation, and each local tensor is computed independently, thus reducing peak memory overhead; simultaneously, shallower propagation paths (such as...) Figure 2 Parallel operations are performed on the local gate tensors in parallel, and the output tensors of each branch are independent of each other, so there is no accumulation during propagation. This helps to alleviate the gradient vanishing problem in chained target tensor networks. As a result, while maintaining the same output physical index dimension as the original linear layer, the global linear transformation can be approximated with fewer gate tensor parameters, significantly reducing the parameter storage size and the number of floating-point operations (FLOPs) during inference / training.

[0134] It should be noted that, as Figure 2As indicated by the dashed arrow in the upper middle section, the output of the target tensor is a high-order tensor (with the shape […,2,2,…,2]). To maintain compatibility with the original Transformer architecture, unless a full network replacement is performed, a reverse reshaping operation is typically needed at the output to restore it to the standard […,4096] shape for connection to subsequent standard network layers. However, if the entire large model network is constructed using the target tensor, the input reshaping only needs to be performed once at the very beginning of the network. Data can maintain its efficient high-order tensor form throughout inter-layer transmission, further eliminating the computational overhead of the reshaping operation.

[0135] Optionally, the process of determining the number of input physical indicators and the number of output physical indicators includes:

[0136] The number of input physical indices and the number of output physical indices of the local gate tensor are calculated according to the following formula:

[0137] N = L1 - L2 + 1

[0138] Where N is the number of local gate tensors, and when L1 is the number of physical indices corresponding to the input feature dimension, L2 is the number of input physical indices (number of input indices) of the local gate tensor; when L1 is the number of physical indices corresponding to the output feature dimension, L2 is the number of output physical indices (number of output indices) of the local gate tensor.

[0139] exist Figure 2 Based on this, see Figure 3 , Figure 3 This is one of the schematic diagrams illustrating the effect of controlling the compression ratio by adjusting the size of the local gate tensor provided by the present invention: When constructing the target tensor, the compression ratio of the model can be achieved by flexibly adjusting the dimensionality of the local gate tensor (the number of input physical indices and / or the number of output physical indices). Figure 3 As shown, the original linear mapping weight matrix has 4096 × 4096 = 2 parameters. 12 ×2 12 In the target tensor, the feature indices of the input data are first reshaped into 12 physical dimensions. Based on this, if the input and output of the local gate tensor (green square) are both set to 11 physical dimensions (i.e., 2^32 physical dimensions), then... 11 ×2 11 To effectively cover all feature dimensions D of the data x (blue square), two local gate tensors need to be configured in the structure. The total number of parameters is then 2. 11 ×2 11 ×2, which is 50% of the original matrix parameters. If the size of the local gate tensor is further reduced, for example, by selecting a local tensor with 10 physical dimensions as the input, then 3 local gate tensors need to be configured, reducing the total number of parameters to 2.10 ×2 10 ×3, which is 18.75% of the original matrix parameters, meaning the compression rate reaches 18.75%; similarly, if a local tensor with 9 physical dimensions is selected as the input, then there are 4 local gate tensors, further reducing the number of parameters to 2. 9 ×2 9 ×4 represents 6.25% of the original matrix parameters; and so on. Therefore, by setting the dimension of the local gate tensor (which directly determines the number of local tensors required), the parameter scale of the model can be precisely controlled to achieve targeted compression effects.

[0140] It should be noted that, although Figure 3 Taking a 4096×4096 square matrix as an example, the target tensor of this invention is also applicable to scenarios where the dimensions of input and output physical indicators are inconsistent (i.e., non-square matrix).

[0141] In non-square matrix computation (where the input feature dimensions and the output feature dimensions are different), the expansion or compression of dimensions can be achieved simply by adjusting the structure of the local gate tensor so that the number of input physical indices is not equal to the number of output physical indices, thereby completing the linear transformation of the non-square matrix.

[0142] exist Figure 2 and Figure 3 Based on this, see Figure 4 , Figure 4 This is the second illustration of the effect of controlling the compression ratio by adjusting the size of the local gate tensor provided by the present invention: the original linear mapping weight matrix has 2048 × 4096 = 2 parameters. 11 ×2 12 In the target tensor, if the input of the local gate tensor (green square) is set to 10 physical dimensions, the output will be 11 physical dimensions, i.e., 2^32 physical dimensions. 10 ×2 11 To effectively cover all feature dimensions D of the data x (blue square), the target tensor needs to contain two local gate tensors. The total number of parameters is then 2. 10 ×2 11 ×2, which is 50% of the original matrix parameters. If the size (number) of the local gate tensor is further reduced, for example, by selecting a local gate tensor with 9 physical dimensions as input and 10 physical dimensions as output, then 3 local gate tensors need to be configured, reducing the total number of parameters to 2. 9 ×2 10 ×3, which is 18.75% of the original matrix parameters, meaning the compression rate reaches 18.75%. Similarly, if a local gate tensor with 8 physical dimensions as input and 9 physical dimensions as output is selected, then there are 4 local gate tensors, further reducing the number of parameters to 2. 8 ×2 9×4 represents 6.25% of the original matrix parameters; and so on. Therefore, by setting the dimension of the local gate tensor (which directly determines the number of local tensors required), the parameter scale of the model can be precisely controlled to achieve targeted compression effects.

[0143] Optionally, concatenating the N local gate tensors using a parallel summation architecture to obtain the target tensor includes:

[0144] The N local gate tensors are concatenated using a parallel summation architecture to obtain the initial tensor;

[0145] The backpropagation algorithm in machine learning is used to initialize the structural parameters of the initial tensor to obtain the target tensor.

[0146] In practical applications, N local gate tensors can be concatenated using a parallel summation architecture to obtain an initial tensor without parameter initialization. Furthermore, the initial tensor's parameters can be initialized using a backpropagation fitting strategy, i.e., the backpropagation algorithm in machine learning, to obtain the target tensor. In this way, the target tensor can reconstruct the weight distribution of the original attention mechanism.

[0147] Optionally, the structural parameters are gate tensor parameters;

[0148] The method employs the backpropagation algorithm from machine learning to initialize the structural parameters of the initial tensor to obtain the target tensor, including:

[0149] Obtain the identity matrix corresponding to the linear mapping weight matrix;

[0150] Input the identity matrix into the initial tensor and output the equivalent weight matrix corresponding to the target tensor;

[0151] Based on the linear mapping weight matrix and the equivalent weight matrix, the loss value is calculated using the Frobenius norm as the loss function.

[0152] Based on the loss value, the gate tensor parameters of the initial tensor are adjusted using the gradient descent method;

[0153] Continue initializing the gate tensor parameters of the adjusted initial tensor until the loss value converges, the initialization is complete, and the target tensor is obtained.

[0154] In practical applications, in order to further enable the initial tensor to reconstruct the weight distribution of the original attention mechanism, this embodiment utilizes the mathematical property that a linear transformation applied to the identity matrix can restore the transformation matrix itself.

[0155] During initialization, an identity matrix with the same dimensions as the original weight matrix (linear mapping weight matrix) is fed into the initial tensor as input data. At this point, the output tensor of the initial tensor represents its current equivalent weight matrix. Subsequently, the linear mapping weight matrix to be compressed in the original model is directly used as the target value, and the difference between the initial tensor output and this target weight matrix is ​​calculated. The loss function uses the Frobenius Norm, as shown below:

[0156] L init =||W1-W2||2 F

[0157] Among them, L init W is the loss value, which is the square of the Frobenius norm; W1 is the linear mapping weight matrix, W2 is the equivalent weight matrix, and F represents the Frobenius norm.

[0158] Furthermore, the above loss function is minimized by the gradient descent algorithm, and the gate tensor parameters (structural parameters) in the initial tensor are updated so that they accurately approximate the original matrix (linear mapping weight matrix) in numerical behavior until the loss value converges, the initialization is completed, and the initialized target tensor is obtained.

[0159] Optionally, when the tensor structure is the target tensor network, the step of using the tensor structure to replace the linear mapping weight matrix in the initial artificial intelligence model to obtain the target artificial intelligence model includes:

[0160] Based on the number of parameters of the feature dimensions involved in the operation in the linear mapping weight matrix, at least two target tensors are constructed, and the target tensors include at least two local gate tensors in parallel.

[0161] The target tensors are layered and connected to obtain the target tensor network;

[0162] Based on the target tensor network, the linear mapping weight matrix in the initial artificial intelligence model is replaced to obtain the initialized artificial intelligence model to be trained;

[0163] Based on the task sample set, the artificial intelligence model to be trained is restored and trained to obtain the target artificial intelligence model. The task sample set includes at least one of the following: natural language processing sample set, logic and mathematical reasoning sample set, code programming sample set, and multimodal content generation sample set.

[0164] In practical applications, at least two target tensors matching the parameter quantity of the feature dimensions involved in the computation in the linear mapping weight matrix can be selected or constructed. These at least two target tensors are then hierarchically connected to form a target tensor network. The construction process of these at least two target tensors is similar to the construction process of the target tensors described above, and will not be repeated here.

[0165] Based on the target tensor network, the linear mapping weight matrix of the initial artificial intelligence model is replaced with the target tensor network to obtain the initialized artificial intelligence model to be trained. Further, the artificial intelligence model to be trained is subjected to recovery training in at least one of the following capabilities: natural language processing, logical and mathematical reasoning, code programming, and multimodal content generation. That is, the artificial intelligence model to be trained is recovered and trained based on the task sample set to obtain the target artificial intelligence model.

[0166] In this embodiment of the invention, a target tensor network is formed by hierarchically connected target tensors to replace the linear mapping weight matrix, thereby achieving structured compression of the artificial intelligence model. This significantly reduces the number of model parameters and memory usage, while also reducing computational complexity and improving inference and training speed.

[0167] Optionally, the step of using a tensor structure to replace the linear mapping weight matrix in the initial artificial intelligence model to obtain the target artificial intelligence model includes:

[0168] Based on the number of parameters of the feature dimensions involved in the operation in the linear mapping weight matrix, at least M1 target tensors are constructed, where M1 is an integer greater than or equal to 3;

[0169] Any M1-M2 target tensors from the at least M1 target tensors are layered and connected to obtain the target tensor network, where M2 is an integer greater than or equal to 1 and less than M1;

[0170] Based on the target tensor network and the target tensors other than the M1-M2 target tensors among the at least M1 target tensors, the linear mapping weight matrix in the initial artificial intelligence model is replaced to obtain the initialized artificial intelligence model to be trained.

[0171] Based on the task sample set, the artificial intelligence model to be trained is restored and trained to obtain the target artificial intelligence model. The task sample set includes at least one of the following: natural language processing sample set, logic and mathematical reasoning sample set, code programming sample set, and multimodal content generation sample set.

[0172] Optionally, the step of retraining the AI ​​model to be trained based on the task sample set to obtain the target AI model includes:

[0173] Obtain the task sample set;

[0174] Based on the task sample set and the loss function, the artificial intelligence model to be trained is restored and trained to obtain the target artificial intelligence model. The loss function is the perplexity loss or cross-entropy loss of the prediction task.

[0175] In practical applications, after parameter initialization is completed, the target tensor and / or target tensor network are formally embedded into the initial artificial intelligence model to replace the original attention mechanism, and end-to-end recovery training is performed.

[0176] During the retraining phase, the selection of the training dataset (task sample set) is highly flexible and can be customized according to the actual application requirements: if the goal is to restore at least one of the model's capabilities such as natural language processing, logical and mathematical reasoning, code programming, and multimodal content generation, a general dialogue or text dataset can be selected; if it is necessary to enhance the model's performance in a specific vertical domain, a corresponding professional domain dataset can be introduced. In addition, the task sample set can be an existing dataset or a specially constructed dataset.

[0177] Accordingly, the loss function used in the optimization process will be adapted according to the dataset type and task objective. Typically, the perplexity (PPL) or cross-entropy loss for the prediction task will be used, as detailed below:

[0178]

[0179] Among them, L train The loss value used to recover the training value of the artificial intelligence model can be the negative log-likelihood loss; t represents the position index of the current word in the sequence in the language task sample; x t x represents the real word at position t in the sequence; <t P(x) represents the context sequence consisting of all lexical units that appear before the t-th lexical unit; t |x <t ) indicates that the artificial intelligence model, given the preceding lexical sequence x, <t Under the condition of predicting the current real word element x <t The probability is denoted by ; log represents taking the logarithm of the predicted probability, which is used to transform the product of probabilities into a summation and enhance the penalty effect for low probability predictions; ∑ represents summing the loss at each position in the sequence; the negative sign indicates that the problem of maximizing the log likelihood is transformed into the problem of minimizing the loss function, which facilitates model training and optimization.

[0180] Furthermore, the gate tensor parameters are fine-tuned through global backpropagation to correct the accuracy loss caused by structural compression, ultimately obtaining a compressed model that combines lightweight design with high performance.

[0181] The following is combined with Figure 5 The performance of the target tensor of the target artificial intelligence model in the method for optimizing artificial intelligence models based on tensor structure provided by the present invention is described.

[0182] In attention mechanism computation, the target structure reduces computational complexity and runtime memory usage compared to existing technologies, while improving parallel computing efficiency.

[0183] To verify this effect, the simulation calculations selected square matrix multiplication with shape (256, 4096) × (4096, 4096) = (256, 4096) and non-square matrix multiplication with shape (256, 4096) × (4096, 16384) = (256, 16384) as test objects. The compression ratio of the tensor structure is controlled by the size of the local gate tensor, and the compression ratio of the matrix multiplication operator method is controlled by the virtual index dimension.

[0184] See Figure 5 , Figure 5 This is a comparison chart showing the effects of the attention mechanism, matrix multiplication operator, and tensor structure implemented in this invention on matrix multiplication: Figure 5 This demonstrates the tensor structure under different compression ratios. Figure 5 (red line in the image) and existing matrix multiplication operator techniques ( Figure 5 The purple, blue, green, and yellow lines represent different physical index dimensions (d) and uncompressed ordinary matrix multiplication. Figure 5 The gray dashed line represents the comparison results of the attention mechanism in three metrics: floating-point operations (Flops), inference time, and memory usage. Figure 5 The feature dimensions of the graph representation in the upper and middle layers that participate in the operation are (256, 4096) × (4096, 4096) = (256, 4096). Figure 5 The lower-level graph representation has a feature dimension of (256, 4096) × (4096, 16384) = (256, 16384). Furthermore, the horizontal axis represents the compression ratio, specifically the compression ratio P2 / P1 of the tensor structure relative to the attention mechanism, where P1 is the computational cost of the attention mechanism and P2 is the computational cost of the tensor structure. The vertical axis represents floating-point operations (Flops), inference time (Time), and memory usage (Memory).

[0185] Experimental results on computational complexity and inference speed show that existing matrix multiplication operator techniques, due to their chained structure requiring sequential computation, limit the utilization of hardware parallel resources. The data in the figure shows that, under most configurations, the floating-point computation and inference time of the matrix multiplication operator method are higher than the baseline of uncompressed ordinary matrix multiplication, and the computational overhead increases further as the physical dimension d decreases. In contrast, tensor structures employ a local gate tensor summation architecture. Figure 5 The red curve in the graph shows that, at all compression rates tested, the floating-point computation and inference time of the tensor structure are lower than those of the matrix multiplication operator method and ordinary matrix multiplication. This indicates that the present invention reduces computational complexity and utilizes the parallel computing capabilities of the GPU to improve matrix operation speed.

[0186] Based on the experimental results of runtime memory usage, the existing matrix multiplication operator technology generates high-dimensional intermediate tensors during the calculation process, which consumes a large amount of video memory. Figure 5 Data shows that the memory usage of the matrix multiplication operator method varies with the compression ratio (controlled by the virtual index dimension) and the physical dimension. Particularly in the large-scale non-square matrix test of (256, 4096) × (4096, 16384), its peak memory usage reaches 10. 4 The megabyte-scale (MB) size far exceeds the uncompressed baseline, making it difficult for this technology to run on memory-constrained devices. Tensor structures optimize the forward propagation logic, reducing the generation of high-dimensional intermediate tensors. Figure 5 The memory usage of the tensor structure shown by the red line remains at a minimum and decreases with increasing compression ratio. Furthermore, combined with the backpropagation-based construction strategy of this technique, the memory overhead during construction is not proportional to the matrix geometry, thus enabling large-scale model compression and reconstruction on general-purpose hardware.

[0187] Figure 6 This is a comparison chart of the training efficiency of different model configurations provided by this invention in a full fine-tuning scenario. Figure 6 This paper presents a comparison of the training efficiency of three different model configurations (first pedestal model, second pedestal model, and target AI model) under fully parameter-tuned experimental settings. In the experimental testing, the open-source AI model Llama 2-7B was selected as the experimental pedestal model (but the method of this invention is not limited to this model). Furthermore, it shows a quantitative comparison of the computational complexity (FLOPs) and video memory usage (VRAM) of the pedestal model at different accuracies (first pedestal model and second pedestal model) compared to the target tensor compression model proposed in this invention.

[0188] Figure 6The horizontal axis represents three model configurations: the first base model, the second base model (with a precision of BF16), and the target artificial intelligence model (BF16) provided by this invention. The precision of the first base model is FL32 (FLOAT32, a single-precision floating-point number), while the precision of the second base model and the target artificial intelligence model is BF16 (BFLOAT16, a 16-bit floating-point number format). Figure 6 The left-hand vertical axis and blue bars represent the computational complexity of a single training step (FLOPs), measured in trillions of floating-point operations (Tera); the right-hand vertical axis and red line graph represent the peak memory usage during training, measured in gigabytes (GB).

[0189] like Figure 6 As shown, a comparison is made between the Llama 2-7B pedestal model (BF16) and the target tensor compression model of this invention (BF16) built based on this model:

[0190] In terms of computational complexity, the base model requires 27.06 Tera FLOPs for single-step training, while the compressed model of this invention significantly reduces the computational cost to 12.97 Tera FLOPs. This indicates that the compression method of this invention substantially reduces the theoretical floating-point operation requirements by approximately 52% at the model architecture level through target tensor structure optimization.

[0191] Regarding video memory usage, although the base model was switched from FL32 to BF16, which reduced the video memory from 53.9 gigabytes (GB) to 27.7GB, the compression model of this invention further reduces the peak video memory to 17.5GB.

[0192] In summary, the comparative data confirms that the method for optimizing artificial intelligence models based on tensor structure proposed in this invention, which uses target tensor compression, can significantly improve the training efficiency of artificial intelligence models represented by Llama 2-7B while maintaining the same training accuracy format (BF16).

[0193] Table 1

[0194]

[0195] Refer to Table 1, which is a comparison table of model performance parameters provided in the embodiments of the present invention. The table shows in detail the model size of the base model (llama2) and the target artificial intelligence model (llama2-TN) using the method of the present invention under various quantization methods (such as float-32, bfloat-16, int-8, and int-4), as well as the peak memory usage comparison data in large-scale, multitask language understanding (MMLU) inference tasks.

[0196] First, looking at the basic properties of the model, as shown in the first column of Table 1, the benchmark model (llama2) contains 6.74B (billion) parameters, and its single inference computation is 726.78B FLOPs. The target AI model of this invention (llama2-TN), through advanced structural optimization, significantly reduces the number of parameters to 3.54B, while simultaneously reducing the inference computation to 457.34B. This reveals the significant advantage of this structural optimization in terms of resource consumption during actual deployment.

[0197] Secondly, the advantages of this invention are more pronounced in terms of specific resource consumption. Under bfloat-16 half-precision, the storage size of the model in this invention (6.67 GB) is reduced by nearly half compared to the pedestal model (12.55 GB). More importantly, its peak memory usage is also significantly reduced from 46.94 GB to 18.59 GB, demonstrating higher runtime efficiency. When using int-8 quantization, the storage size (3.30 GB) and peak memory usage (18.59 GB) of the model in this invention are also far superior to the pedestal model under the same quantization configuration.

[0198] In the extreme compression scenario of int-4 quantization, the technical advantages of this invention reach their peak. The compression model of this invention not only compresses the storage size to less than 1GB (0.96 GB), but also maintains a low peak GPU memory usage of 18.59 GB. This means that models applying the technology of this invention can easily adapt to edge computing devices, mobile terminals, or IoT devices with extremely limited memory resources, significantly reducing the hardware threshold for deploying large-scale language models on the edge, and verifying the significant technical effects of this invention in both computational efficiency and storage space optimization.

[0199] In stark contrast, while the base model's storage size also decreased to 1.63 GB under int-4 quantization, its peak memory usage rebounded to 93.87 GB. This may be due to the memory overhead of dequantization operations or specific operators. This data fully demonstrates that the compression method of this invention not only achieves extreme optimization in storage but also ensures stable and efficient memory performance during inference under low-bit-width quantization.

[0200] The present invention provides a method for optimizing artificial intelligence models based on tensor structures. Utilizing a target tensor with a local gate tensor parallel summation architecture, it overcomes the limitation of traditional target tensor networks requiring sequential, step-by-step contraction calculations. Instead, it decomposes the originally massive overall computation into multiple independent and concurrent local tensor operations. This highly parallel structure effectively utilizes the massively parallel computing resources of GPUs, improving inference speed. Simultaneously, this structure reduces floating-point operations and runtime memory usage, achieving comprehensive compression of the model in terms of parameter size, computational complexity, and inference time. Furthermore, this parallel structure avoids the training difficulties caused by excessively deep network layers, allowing the artificial intelligence model to directly utilize the backpropagation algorithm for stable parameter initialization, no longer limited by the resource bottleneck caused by global decomposition of extremely large matrices.

[0201] The apparatus for optimizing artificial intelligence models based on tensor structures provided by the present invention will be described below. The apparatus for optimizing artificial intelligence models based on tensor structures described below and the method for optimizing artificial intelligence models based on tensor structures described above can be referred to in correspondence.

[0202] Figure 7 This is a schematic diagram of the device for optimizing artificial intelligence models based on tensor structures provided by the present invention, as shown below. Figure 7 As shown, the device for optimizing an artificial intelligence model based on tensor structure includes:

[0203] Replacement module 701 is configured to use a tensor structure to replace the linear mapping weight matrix in the initial artificial intelligence model to obtain a target artificial intelligence model. The tensor structure includes a target tensor and / or a target tensor network. The target artificial intelligence model is used to process a target task, which includes at least one of natural language processing, logic and mathematical reasoning, code programming, and multimodal content generation.

[0204] The device for optimizing artificial intelligence models based on tensor structures provided by this invention replaces the linear mapping weight matrix in the Transformer architecture of artificial intelligence models with a target tensor and / or a target tensor network. It reconstructs high-dimensional fully connected matrix operations into a parallel summation of multiple local gate tensors, leveraging the independence of the local gate tensors to achieve parallelization of the computation process. This structure, while eliminating redundant parameters and significantly compressing the model size, effectively shortens inference latency by reducing floating-point operations and dynamic memory usage, thereby significantly accelerating model inference on general-purpose hardware, i.e., improving task processing efficiency.

[0205] Optionally, when the tensor structure is the target tensor, the replacement module 701 is specifically configured as follows:

[0206] Based on the number of parameters of the feature dimensions involved in the operation in the linear mapping weight matrix, the target tensor is constructed, and the target tensor includes at least two local gate tensors in parallel.

[0207] Based on the target tensor, the linear mapping weight matrix in the initial artificial intelligence model is replaced to obtain the initialized artificial intelligence model to be trained;

[0208] Based on the task sample set, the artificial intelligence model to be trained is restored and trained to obtain the target artificial intelligence model. The task sample set includes at least one of the following: natural language processing sample set, logic and mathematical reasoning sample set, code programming sample set, and multimodal content generation sample set.

[0209] Optionally, the feature dimensions involved in the operation in the linear mapping weight matrix include input feature dimensions and output feature dimensions;

[0210] The replacement module 701 is specifically configured as follows:

[0211] Determine the dimensions of physical indicators;

[0212] Based on the physical index dimension, determine the number of physical indices corresponding to the input feature dimension and the number of physical indices corresponding to the output feature dimension;

[0213] Based on the number of physical indicators corresponding to the input feature dimension, the number of physical indicators corresponding to the output feature dimension, and the target compression ratio, the number of local gate tensors N and the number of input and output physical indicators of the local gate tensor are determined with the physical indicator dimension as the input physical indicator dimension and the output physical indicator dimension of the local gate tensor. The number of input physical indicators is less than the number of physical indicators corresponding to the input feature dimension, and the number of output physical indicators is less than the number of physical indicators corresponding to the output feature dimension.

[0214] The target tensor is obtained by concatenating the N local gate tensors according to a parallel summation architecture.

[0215] Optionally, the device further includes a processing module configured to:

[0216] The target task is encoded based on the input layer of the target artificial intelligence model to obtain the input vector corresponding to the target task;

[0217] Using the target tensor, the input vector is reshaped based on the physical index dimension to obtain a higher-order tensor corresponding to the number of physical indices corresponding to the input feature dimension;

[0218] By performing tensor contraction operations on the higher-order tensors using each of the local gate tensors, the branch output tensors of each of the local gate tensors are obtained;

[0219] The output tensors of each branch are summed and normalized according to the number of local gate tensors to obtain the target output tensor.

[0220] The target output tensor is decoded based on the output layer of the target artificial intelligence model to obtain the processing result of the target task.

[0221] Optionally, the replacement module 701 is specifically configured as follows:

[0222] For any one of the input feature dimensions and the output feature dimensions, the dimension is transformed according to the following formula to obtain the number of physical indicators corresponding to the dimension:

[0223] D=d L

[0224] Where D is the dimension, d is the physical index dimension, and L is the number of physical indices corresponding to the dimension.

[0225] Optionally, the replacement module 701 is specifically configured as follows:

[0226] The number of input physical indices and the number of output physical indices of the local gate tensor are calculated according to the following formula:

[0227] N = L1 - L2 + 1

[0228] Where N is the number of local gate tensors, L1 is the number of physical indices corresponding to the input feature dimension, and L2 is the number of input physical indices of the local gate tensor, or L 1r L1 represents the number of physical indices corresponding to the output feature dimension, and L2 represents the number of output physical indices of the local gate tensor.

[0229] Optionally, the replacement module 701 is specifically configured as follows:

[0230] The N local gate tensors are concatenated using a parallel summation architecture to obtain the initial tensor;

[0231] The backpropagation algorithm in machine learning is used to initialize the structural parameters of the initial tensor to obtain the target tensor.

[0232] Optionally, the structural parameters are gate tensor parameters;

[0233] The building module is specifically configured as follows:

[0234] Obtain the identity matrix corresponding to the linear mapping weight matrix;

[0235] Input the identity matrix into the initial tensor and output the equivalent weight matrix corresponding to the target tensor;

[0236] Based on the linear mapping weight matrix and the equivalent weight matrix, the loss value is calculated using the Frobenius norm as the loss function.

[0237] Based on the loss value, the gate tensor parameters of the initial tensor are adjusted using the gradient descent method;

[0238] Continue initializing the gate tensor parameters of the adjusted initial tensor until the loss value converges, the initialization is complete, and the target tensor is obtained.

[0239] Optionally, when the tensor structure is the target tensor network, the replacement module 701 is specifically configured as follows:

[0240] Based on the number of parameters of the feature dimensions involved in the operation in the linear mapping weight matrix, at least two target tensors are constructed, and the target tensors include at least two local gate tensors in parallel.

[0241] The target tensors are layered and connected to obtain the target tensor network;

[0242] Based on the target tensor network, the linear mapping weight matrix in the initial artificial intelligence model is replaced to obtain the initialized artificial intelligence model to be trained;

[0243] Based on the task sample set, the artificial intelligence model to be trained is restored and trained to obtain the target artificial intelligence model. The task sample set includes at least one of the following: natural language processing sample set, logic and mathematical reasoning sample set, code programming sample set, and multimodal content generation sample set.

[0244] Optionally, the replacement module 701 is specifically configured as follows:

[0245] Obtain the task sample set;

[0246] Based on the task sample set and the loss function, the artificial intelligence model to be trained is restored and trained to obtain the target artificial intelligence model. The loss function is the perplexity loss or cross-entropy loss of the prediction task.

[0247] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 8 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a method for optimizing an artificial intelligence model based on a tensor structure. This method includes: replacing the linear mapping weight matrix in an initial artificial intelligence model with a tensor structure to obtain a target artificial intelligence model. The tensor structure includes a target tensor and / or a target tensor network. The target artificial intelligence model is used to process a target task, which includes at least one of natural language processing, logical and mathematical reasoning, code programming, and multimodal content generation.

[0248] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0249] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the method for optimizing an artificial intelligence model based on a tensor structure provided by the above methods. The method includes: using a tensor structure to replace the linear mapping weight matrix in the initial artificial intelligence model to obtain a target artificial intelligence model. The tensor structure includes a target tensor and / or a target tensor network. The target artificial intelligence model is used to process a target task. The target task includes at least one of natural language processing, logical and mathematical reasoning, code programming, and multimodal content generation.

[0250] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for optimizing an artificial intelligence model based on a tensor structure provided by the above methods. The method includes: replacing the linear mapping weight matrix in an initial artificial intelligence model with a tensor structure to obtain a target artificial intelligence model, wherein the tensor structure includes a target tensor and / or a target tensor network, and the target artificial intelligence model is used to process a target task, wherein the target task includes at least one of natural language processing, logical and mathematical reasoning, code programming, and multimodal content generation.

[0251] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0252] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0253] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for optimizing an artificial intelligence model based on a tensor structure, characterized in that, include: A target artificial intelligence model is obtained by replacing the linear mapping weight matrix in the initial artificial intelligence model with a tensor structure. The tensor structure includes a target tensor and / or a target tensor network. The target artificial intelligence model is used to process a target task, which includes at least one of natural language processing, logical and mathematical reasoning, code programming, and multimodal content generation. When the tensor structure is the target tensor, the step of replacing the linear mapping weight matrix in the initial artificial intelligence model with the tensor structure to obtain the target artificial intelligence model includes: Based on the number of parameters of the feature dimensions involved in the operation in the linear mapping weight matrix, the target tensor is constructed, and the target tensor includes at least two local gate tensors in parallel. Based on the target tensor, the linear mapping weight matrix in the initial artificial intelligence model is replaced to obtain the initialized artificial intelligence model to be trained; Based on the task sample set, the artificial intelligence model to be trained is restored and trained to obtain the target artificial intelligence model. The task sample set includes at least one of the following: natural language processing sample set, logic and mathematical reasoning sample set, code programming sample set, and multimodal content generation sample set. The feature dimensions involved in the calculation in the linear mapping weight matrix include the input feature dimension and the output feature dimension; The construction of the target tensor based on the number of parameters of the feature dimensions involved in the operation in the linear mapping weight matrix includes: Determine the dimensions of physical indicators; Based on the physical index dimension, determine the number of physical indices corresponding to the input feature dimension and the number of physical indices corresponding to the output feature dimension; Based on the number of physical indicators corresponding to the input feature dimension, the number of physical indicators corresponding to the output feature dimension, and the target compression ratio, the number of local gate tensors N and the number of input and output physical indicators of the local gate tensor are determined with the physical indicator dimension as the input physical indicator dimension and the output physical indicator dimension of the local gate tensor. The number of input physical indicators is less than the number of physical indicators corresponding to the input feature dimension, and the number of output physical indicators is less than the number of physical indicators corresponding to the output feature dimension. The target tensor is obtained by concatenating the N local gate tensors according to a parallel summation architecture.

2. The method for optimizing artificial intelligence models based on tensor structures according to claim 1, characterized in that, After replacing the linear mapping weight matrix in the initial artificial intelligence model with a tensor structure to obtain the target artificial intelligence model, the process further includes: The input layer of the target artificial intelligence model encodes the target task to obtain the input vector corresponding to the target task. Using the target tensor, the input vector is reshaped based on the physical index dimension to obtain a higher-order tensor corresponding to the number of physical indices corresponding to the input feature dimension; By performing tensor contraction operations on the higher-order tensors using each of the local gate tensors, the branch output tensors of each of the local gate tensors are obtained; The output tensors of each branch are summed and normalized according to the number of local gate tensors to obtain the target output tensor. The target output tensor is decoded based on the output layer of the target artificial intelligence model to obtain the processing result of the target task.

3. The method for optimizing artificial intelligence models based on tensor structures according to claim 1, characterized in that, The step of determining the number of physical indicators corresponding to the input feature dimension and the number of physical indicators corresponding to the output feature dimension based on the physical indicator dimension includes: For any one of the input feature dimensions and the output feature dimensions, the dimension is transformed according to the following formula to obtain the number of physical indicators corresponding to the dimension: D = d L Where D is the dimension, d is the physical index dimension, and L is the number of physical indices corresponding to the dimension.

4. The method for optimizing artificial intelligence models based on tensor structures according to claim 1, characterized in that, The process of determining the number of input physical indicators and the number of output physical indicators includes: The number of input physical indices and the number of output physical indices of the local gate tensor are calculated according to the following formula: N = L1 - L2 + 1 Where N is the number of local gate tensors; L1 is the number of physical indices corresponding to the input feature dimension and L2 is the number of input physical indices of the local gate tensor, or L1 is the number of physical indices corresponding to the output feature dimension and L2 is the number of output physical indices of the local gate tensor.

5. The method for optimizing artificial intelligence models based on tensor structures according to claim 1, characterized in that, The step of concatenating the N local gate tensors according to a parallel summation architecture to obtain the target tensor includes: The N local gate tensors are concatenated using a parallel summation architecture to obtain the initial tensor; The backpropagation algorithm in machine learning is used to initialize the structural parameters of the initial tensor to obtain the target tensor.

6. The method for optimizing artificial intelligence models based on tensor structures according to claim 5, characterized in that, The structural parameters are gate tensor parameters; The method employs the backpropagation algorithm from machine learning to initialize the structural parameters of the initial tensor to obtain the target tensor, including: Obtain the identity matrix corresponding to the linear mapping weight matrix; Input the identity matrix into the initial tensor and output the equivalent weight matrix corresponding to the initial tensor; Based on the linear mapping weight matrix and the equivalent weight matrix, the loss value is calculated using the Frobenius norm as the loss function. Based on the loss value, the gate tensor parameters of the initial tensor are adjusted using the gradient descent method; Continue initializing the gate tensor parameters of the adjusted initial tensor until the loss value converges, the initialization is complete, and the target tensor is obtained.

7. The method for optimizing artificial intelligence models based on tensor structures according to claim 1, characterized in that, In the case where the tensor structure is the target tensor network, the step of replacing the linear mapping weight matrix in the initial artificial intelligence model with the tensor structure to obtain the target artificial intelligence model includes: Based on the number of parameters of the feature dimensions involved in the operation in the linear mapping weight matrix, at least two target tensors are constructed, and the target tensors include at least two local gate tensors in parallel. The target tensors are layered and connected to obtain the target tensor network; Based on the target tensor network, the linear mapping weight matrix in the initial artificial intelligence model is replaced to obtain the initialized artificial intelligence model to be trained; Based on the task sample set, the artificial intelligence model to be trained is restored and trained to obtain the target artificial intelligence model. The task sample set includes at least one of the following: natural language processing sample set, logic and mathematical reasoning sample set, code programming sample set, and multimodal content generation sample set.

8. The method for optimizing an artificial intelligence model based on tensor structure according to any one of claims 1-7, characterized in that, The step of restoring and training the AI ​​model to be trained based on the task sample set to obtain the target AI model includes: Obtain the task sample set; Based on the task sample set and the loss function, the artificial intelligence model to be trained is restored and trained to obtain the target artificial intelligence model. The loss function is the perplexity loss or cross-entropy loss of the prediction task.

Images