Large model tensor database construction method and system

Abstract

The invention relates to the field of data processing, in particular to a large-model tensor database construction method and system.The method comprises the steps that tensors are captured and clustered according to semantics, sub-modes in the clusters are mined through a Gaussian mixture model (GMM), and a shared base is constructed; during online processing, a new tensor is decomposed into a composite representation formed by a mode probability and a core tensor through multi-base projection. According to the representation, structural information is reserved while data is greatly compressed, and efficient structured query and analysis are achieved. According to the method, the multi-structure sub-modes in the tensor cluster are mined by adopting the Gaussian mixture model, the shared bases are constructed for multi-base projection, and the composite core representation is generated to realize compression and analysis.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, in particular to a tensor database construction method and system for large models. BACKGROUND

[0002] With the rapid development of large language models (LLMs), understanding their internal mechanisms and explaining their behavior patterns have become key research focuses in the field of artificial intelligence. During the inference process of LLMs, intermediate layer tensors such as attention weights and hidden states are generated. By deeply studying the structure of these tensors, the internal computing patterns that the model relies on and can reuse when processing different tasks can be revealed, providing a theoretical basis for the explainability, compression, and optimization of the model.

[0003] In the prior art, a common analysis method is to first collect the intermediate layer tensors generated by the model when processing a batch of semantically similar inputs, and group them into a cluster as a behavior sample set of the model for a specific task. How to effectively mine one or more representative structural patterns from these complex high-dimensional tensor clusters. A feasible technical means is to extract a feature vector that can represent the internal structure of each tensor, and then apply an unsupervised clustering algorithm to the set of feature vectors to discover different structural patterns.

[0004] Among them, Gaussian Mixture Model (GMM) is one of the commonly used algorithms, because it can model data generated by multiple different distributions. However, in the context of LLM tensor analysis, direct application of GMM has inherent limitations. First, the cluster center (mean vector) learned by GMM can only represent the average form of one structural pattern, and it is easily affected by noise and outliers within the cluster, resulting in a less typical pattern represented. Secondly, the mean vector itself is an abstract statistical quantity and cannot directly provide a structured basis for the pattern that is simple, efficient and can be used for decomposition and reconstruction, making it difficult to apply the mined pattern to subsequent tasks such as tensor compression, efficient representation, or deep analysis. SUMMARY

[0005] In view of the difficulty of applying the cluster center of the above GMM to tensor compression, efficient representation or deep analysis problem, in the first aspect, the application proposes a tensor database construction method of large model, comprising: obtaining a plurality of intermediate layer tensors generated by a large language model in an inference process, and context meta information corresponding to the plurality of intermediate layer tensors; dividing the plurality of intermediate layer tensors into different tensor clusters based on the context meta information; for any tensor cluster, extracting a structural feature vector along at least one dimension of the tensor contained therein, the structural feature vector representing the covariance structure of the corresponding tensor in the dimension; using a Gaussian mixture model to cluster the structural feature vector set in the tensor cluster to obtain a plurality of structural sub-patterns; constructing a dedicated shared basis for each structural sub-pattern, comprising: calculating the probability weighted average of the covariance matrix of the tensor corresponding to the structural sub-pattern, and performing eigenvalue decomposition on the obtained weighted average covariance matrix to obtain the shared basis; in response to receiving a new intermediate layer tensor, projecting it onto the plurality of shared bases corresponding to the tensor cluster to which it belongs to obtain a group of core tensors; and combining the core tensors with the attribution probability information of the structural sub-patterns to obtain a composite core representation of the new intermediate layer tensor.

[0006] Unlike the prior art which usually directly stores the original intermediate layer tensors of the large language model or uses a single global compression method (such as PCA), the application proposes a new structured analysis and compression framework. By first clustering by semantics, then using a Gaussian mixture model to mine structural sub-patterns in the cluster and constructing a dedicated shared basis for each pattern, the tensor is finally decomposed into the attribution probability of the pattern and the composite representation of the core tensor. This method not only greatly compresses the storage volume of the tensor and saves hardware costs, but more importantly, it explicitly quantifies the implicit and difficult-to-use structural information in the tensor into discrete pattern attribution, making it possible for subsequent efficient structured query and deep analysis, and solving the problem that traditional methods cannot balance high compression rate and structural information preservation.

[0007] Further, the division into different tensor clusters comprises: dividing the intermediate layer tensors having the same model source, level index and tensor type, and the semantic vector similarity of the input text being higher than a preset threshold, into the same tensor cluster.

[0008] Compared with mixed analysis of all tensors or division only by level, the present application pre-clusters by model source, level, type and input text semantic similarity, ensuring that the subsequent structural analysis is performed on a highly similar function and context tensor set. This refined division avoids the noise and bias brought by mixed analysis of heterogeneous data, significantly improves the accuracy and purity of the Gaussian mixture model mining structural sub-patterns, makes the shared basis constructed for each cluster more representative, and thus improves the overall compression effect and analysis accuracy.

[0009] Further, the extracting the structural feature vector comprises: expanding the tensor along the dimension to obtain a two-dimensional matrix; calculating a covariance matrix of the two-dimensional matrix; and vectorizing the covariance matrix.

[0010] The application discloses that the structural feature vector is extracted by calculating the covariance matrix. Compared with directly analyzing original tensor values, the covariance matrix can more essentially and stably represent internal data correlation and distribution structure of the tensor in a specific dimension. This makes clustering of the structural mode no longer disturbed by specific value size, but focuses on inherent statistical characteristics, so that common structural sub-modes can be more accurately mined, and universality and robustness of the method to tensors with different value ranges are enhanced.

[0011] Further, the clustering by using the Gaussian mixture model further comprises: automatically determining the number of the structural sub-modes by using Bayesian information criterion or Akaike information criterion within a preset range.

[0012] Compared with a traditional method which needs to manually preset a clustering number (namely, K value) according to experience, the application automatically determines the number of the structural sub-modes by using Bayesian information criterion (BIC) or Akaike information criterion (AIC). This avoids subjectivity and tediousness of manual parameter adjustment, enables the model to adaptively find an optimal mode division according to complexity of data itself, and thus improves automation degree, objectivity and scalability of the whole analysis process.

[0013] Further, the probability weighted average value is obtained by weighting posterior probabilities of tensors belonging to corresponding structural sub-modes; and the posterior probabilities are calculated based on the Gaussian mixture model.

[0014] The application discloses a calculation mode of weighted average of the covariance matrix by using the posterior probability. This mode ensures that, when the shared basis is constructed, tensors which typically belong to the structural sub-mode can be more considered, and influence of abnormal or marginal tensors is reduced. Compared with simple arithmetic average, this weighted mode can construct a shared basis which is more pure and can represent core characteristics of the mode, so as to improve accuracy and compression efficiency of projection decomposition.

[0015] Further, the shared basis is composed of eigenvectors corresponding to the first M largest eigenvalues of the weighted average covariance matrix, wherein M is a preset decomposition rank. M is a preset decomposition rank.

[0016] Further, the decomposition rank makes energy of the reserved eigenvalue account for more than 99% of total energy.

[0017] ​Further, when the belonging probability of a structural sub-pattern is lower than a preset probability threshold, the calculation or storage of the core tensor corresponding to the structural sub-pattern is stopped.

[0018] Further, the composite core representation is a structured object, including: an identifier of the new intermediate layer tensor, an identifier of the tensor cluster to which it belongs, belonging probability information of the structural sub-pattern, and a pointer to the storage location of a set of core tensors.

[0019] In a second aspect, the present application provides a tensor database construction system of a large model, comprising a processor and a memory, the memory storing computer program instructions which, when executed by the processor, implement a tensor database construction method of a large model of the present application.

[0020] The technical effects of the present application are: The present application proposes an innovative structured compression and analysis method to solve the problems of large storage of intermediate layer tensors in large language models and difficulty in utilizing structural information. The core innovation is to extract the implicit and continuous structural features in the tensor into discrete and identifiable structural sub-patterns through semantic clustering and Gaussian mixture model (GMM) secondary mining. Then, by constructing a dedicated shared basis for each pattern and performing multi-basis projection, the original large tensor is decomposed into a composite representation containing pattern belonging probability and multiple compact core tensors. This method not only achieves high-rate data compression, but also decouples and explicitly represents the structural information of the tensor for the first time, laying a foundation for efficient structured query and deep analysis. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is a flow chart of a tensor database construction method of a large model according to an embodiment of the present application; Figure 2 is a bar chart of the size distribution of a tensor cluster according to an embodiment of the present application; Figure 3 is a BIC curve diagram of GMM mode number selection according to an embodiment of the present application; Figure 4 is a histogram of tensor reconstruction error distribution according to an embodiment of the present application; Figure 5 is a structural block diagram of a tensor database construction system of a large model according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0024] An example of a method for constructing a tensor database for large models: like Figure 1 As shown, a method for constructing a tensor database for a large model according to the present invention includes: S1. Context-aware tensor capture and semantically driven tensor cluster construction.

[0025] In this embodiment, the first step is to obtain the original intermediate layer tensors as the analysis object from the inference process of the Large Language Model (LLM) and organize them with contextual information to lay the data foundation for subsequent pattern mining.

[0026] S1.1. Non-intrusive tensor capture of LLM intermediate layers using frame hooks.

[0027] In one embodiment, hooks can be set in large language model inference frameworks (such as PyTorch and TensorFlow) to non-intrusively capture intermediate layer tensors of a specified level during the model's forward propagation. For example, the attention weight tensor output by the self-attention module of a specific Transformer layer, or the hidden state tensor output by the feedforward network (FFN), can be captured. The captured tensor It is temporarily stored in a high-throughput temporary message queue (such as Kafka or RabbitMQ) and awaits subsequent batch processing.

[0028] S1.2 Bind structured contextual information, including semantic vectors, to tensors.

[0029] Capturing tensors At the same time, the system captures and binds its complete contextual metadata. This metadata is a structured JSON object used to describe tensor information, and it contains at least the following fields: tensor_id: A globally unique tensor identifier; model_id: The name and version of the model that generated this tensor, such as "gpt-4"; layer_index: the layer index where the tensor comes from, e.g. "encoder.layer.10"; tensor_type: the type of the tensor, e.g. "attention_weights" or "hidden_state"; head_index: when tensor_type is attention_weights, this field is used to identify the specific attention head index; input_id: the unique identifier of the input data that triggers the generation of this tensor; input_text: the original input text; input_semantic_vector: the semantic vector representation of the input text. This vector can be computed in real-time by a separate, lightweight sentence encoding model (e.g. Sentence-BERT), or it can be retrieved from a pre-computed vector library.

[0030] S1.3, according to the semantic vector cosine similarity threshold, dynamically divide the tensor into clusters.

[0031] The background batch processing service consumes tensor data with meta information from the message queue. Based on the meta information, the tensor is dynamically divided into different tensor clusters . The grouping rule is configurable. As a typical grouping rule, tensors from the same model_id, the same layer_index, the same tensor_type, and whose input_semantic_vector has a cosine similarity in the vector space higher than the pre-set threshold are divided into the same cluster.

[0032] In this embodiment, the pre-set threshold is preferably 0.85-0.95. If the threshold is lower than 0.85, it may group tensors corresponding to inputs with large semantic differences into a cluster, resulting in a too mixed cluster structure pattern, which is not conducive to subsequent purification of the shared base; if the threshold is higher than 0.95, the grouping condition is too strict, which may result in a too small cluster size, and the structure pattern with generalization ability cannot be effectively learned. Therefore, controlling the threshold around 0.9 can ensure the consistency of the semantic consistency in the cluster while ensuring that there are enough data samples for pattern mining. Each tensor cluster represents a group of behavior samples generated by the model at a specific position when processing a similar task.

[0033] For example Figure 2As shown, the figure illustrates the five largest tensor clusters formed after grouping by hierarchy and semantic similarity, along with the number of tensors they contain. For example, cluster_000, originating from layer 10, contains approximately 120 tensors and is the largest cluster. This figure visually demonstrates the effectiveness of the grouping mechanism, namely, its ability to aggregate a large number of discrete tensors into behavioral sample sets of varying sizes suitable for subsequent analysis.

[0034] S2. Apply Gaussian mixture model to mine multi-structure patterns within clusters and generate pattern-specific shared bases.

[0035] This step aims to address the limitations of the unimodal distribution assumption and is responsible for processing data from each tensor cluster. In this process, multiple discrete structural sub-patterns that may coexist are automatically discovered, and a unique, highly purified shared basis vector is constructed for each sub-pattern.

[0036] S2.1 Calculate and quantize the covariance matrix along each dimension to extract structural features.

[0037] For any data cluster obtained in step S1 and what it contains Tensor For each tensor, the first For each dimension, a feature vector representing its structural characteristics in that dimension needs to be extracted. For a third-order tensor, the dimension index... .

[0038] First, calculate the tensor. Along the first Dimensional expansion ( The matrix obtained after ) Then calculate the covariance matrix of this matrix. The covariance matrix can capture the degree of variation and correlation of data in different directions, thus reflecting the structural pattern inside the tensor, specifically: ; in Tensor In the The covariance matrix in dimension 1; Representation matrix The transpose of . This covariance matrix contains tensors. In the Complete second-order structure information on the dimension.

[0039] To facilitate subsequent cluster analysis, this high-dimensional symmetric covariance matrix needs to be... Vectorization yields a one-dimensional structural feature vector. Specifically, its upper triangular (or lower triangular) portion can be taken and flattened by row (or column) to form a one-dimensional vector.

[0040] S2.2 Utilize GMM to perform unsupervised clustering of structural features and adaptively determine the number of patterns.

[0041] Collect all tensors within the cluster in dimension The set of structural feature vectors on In this embodiment, to better verify the effectiveness of the subsequently constructed database, it is assumed that the vector set does not follow a single distribution, but is a mixture of multiple potential, discrete structural sub-patterns. Therefore, a Gaussian mixture model (Gaussian Mixture Model) is applied. , Perform unsupervised clustering on the vector set.

[0042] GMM is able to fit data because... The model is generated by mixing different Gaussian distributions. In this scenario, each Gaussian component ( ) precisely corresponds to a potential, discrete structural sub-pattern ( By maximizing expectation ( The algorithm trains the GMM, and the model can automatically learn the mean of each Gaussian component. (Representative Pattern) (Central typical structure), covariance (Representative Pattern) (Internal structural variation range) and mixed weights (Representative Model) (Frequency of occurrence in the cluster).

[0043] Optimal number of clusters (i.e., the number of modes) can be set within a preset range (e.g.) The search was performed using the Bayesian information criterion (...). ) or Akaike Information Guidelines ( (This will be automatically determined.) When the value is too small, meaningful sub-patterns cannot be distinguished, leading to underfitting of the model; when... When the value is too large, noise may also be fitted as an independent pattern, leading to model overfitting. The BIC / AIC criterion can effectively balance model complexity and goodness of fit, achieving adaptive discovery of the number of patterns. The training process of the GMM described above is a well-known technique, and its specific implementation will not be elaborated here.

[0044] After GMM training is complete, for any tensor within the cluster The model can calculate its structural feature vector. belong any one of the different structural sub-patterns posterior probability This probability vector provides soft information about the structural attribution of the tensor .

[0045] As shown in Figure 3 , it uses Bayesian Information Criterion (BIC) to adaptively find the optimal number of modes (K) . In the figure, the X-axis is the number of components (i.e. the number of modes) that GMM tries, and the Y-axis is the corresponding BIC value. According to the principle of selecting the model that minimizes the BIC value, when the number of components is 4, the BIC value reaches the lowest point, that is, for the current analyzed tensor cluster, the system automatically discovers that there are 4 main structural sub-patterns coexisting inside it.

[0046] S2.3, construct a weighted covariance matrix based on the posterior probability of GMM, and decompose to generate the shared basis of each mode.

[0047] For each structural sub-pattern discovered by GMM , construct its own dedicated, highly purified shared basis. Specifically, calculate the probability-weighted average covariance matrix in dimension . This formula can make tensors that are highly consistent with the structure of the mode contribute more when constructing the representative covariance matrix of this mode, while the contribution of inconsistent tensors is suppressed, specifically: ; where denotes the posterior probability of tensor belonging to mode ; denotes the covariance matrix of tensor ; denotes the total number of tensors in the cluster; denotes a very small positive number used to avoid the case where the denominator is 0, which can be taken as the empirical value 1e-6 in this embodiment.

[0048] This finally generated can accurately depict the unique structure of mode , avoiding pollution by tensors of other modes. When the structural characteristics of a tensor are closer to the center of mode , its posterior probability value is larger, and its proportion in the weighted average is also larger, thus dominating the formation of the tensor.

[0049] Finally, the weighted average covariance matrix of each mode is respectively decomposed into eigenvalues and eigenvectors. The eigenvectors corresponding to the first largest eigenvalues are selected to form the shared factor matrix (i.e. shared basis) of the mode in this dimension. where is the dimension of the shared basis, is the rank of the decomposition.

[0050] In this embodiment, the rank of the decomposition affects the balance between compression rate and fidelity. Preferably, the value of is set so that the energy of the retained eigenvalues accounts for or more of the total energy, or it is taken as a tunable hyperparameter, for example, with a value of or .

[0051] As shown in Figure 4 , this histogram shows the distribution of relative errors between the original tensor and the reconstructed tensor after the tensor is compressed by multi-base projection and then reconstructed using the core tensor and shared basis. The chart shows that the reconstruction errors of most tensors are concentrated between 0.5-0.6, with an average error of 0.4536. This value is a key indicator of information fidelity, and its size can be controlled by adjusting hyperparameters such as the rank Rn mentioned in S2.3, thereby achieving a balance between compression efficiency and data accuracy.

[0052] S3, construct a hybrid storage database to store the tensor cluster, structure mode and shared basis in a structured manner.

[0053] This step is responsible for the structured and queryable storage of the shared factor matrix calculated in step S2, forming the knowledge base of the entire system. The database uses a hybrid storage scheme, for example, using a document database (such as MongoDB) to store metadata and model information, and using object storage (such as "S3 path") or high-performance key-value storage (such as RocksDB) to store numerical data.

[0054] The data model is designed as follows: TensorClusters collection: stores the definition information of the tensor cluster; cluster_id: unique identifier of the cluster; model_id, layer_index, tensor_type: meta information defining the cluster; semantic_centroid: the mean of all input_semantic_vector in the cluster, representing the semantic center of the cluster; gmm_model_path_dim_n: the path to the GMM model for this dimension of the cluster (e.g. "S3 path"); StructureModes collection: stores the structural sub-mode information discovered in each cluster; mode_id: unique identifier of the mode; cluster_id: the cluster ID it belongs to; dimension: the corresponding dimension index ; gmm_component_index: the corresponding GMM component index; mixing_weight: the frequency of the mode appearing in the cluster (i.e. the mixing weight of the GMM ); shared_basis_id: the ID pointing to the mode-specific shared basis; SharedBases storage: stores the specific shared factor matrices (basis vectors) in key-value pairs; Key: shared basis ID, such as "basis_pqr"; Value: serialized NumPy array or HDF5 file containing the shared factor matrix set of the mode in all dimensions ; S4, real-time decomposition and compressed storage of new tensors based on multi-base projection and composite representation.

[0055] This step is the online service part of the system, responsible for real-time and efficient decomposition and storage of newly generated tensors.

[0056] S4.1, locate the corresponding cluster according to the meta information of the new tensor, and load the GMM model to determine the mode attribution probability.

[0057] When the online service receives a new tensor and its meta information , first find the tensor cluster cluster_abc it belongs to in the TensorClusters collection according to ; then load the GMM model for each dimension corresponding to the cluster (e.g. gmm_abc_d1.pkl); then, real-time extract the structural feature vector of each dimension ​The feature vector is then input into the corresponding GMM model to calculate... The posterior probability vectors belonging to each structural sub-pattern within this cluster .

[0058] S4.2. Project in parallel to all mode bases to generate a composite representation containing probability and core tensor and store it in the library.

[0059] According to the output of GMM, the service retrieves all data from StructureModes and SharedBases at once. The set of shared factor matrices corresponding to each pattern. Execute separately The parallel projective decomposition involves projecting the decomposition onto the shared base space of each pattern, resulting in a set of... One core tensor . No. The calculation of each core tensor is based on the inverse process of Tucker decomposition, specifically: ; in express In mode The core tensor under the base; Indicates along the first Matrix product of modulo; Representation pattern In dimensions Transpose of the shared base on.

[0060] The system does not store large raw tensors Instead, it generates and stores a highly condensed composite core representation ( This representation is a structured object containing the following information: entry_id: The unique ID of this stored entry; original_tensor_id: original tensor ID; cluster_id: The ID of the cluster to which it belongs; mode_probabilities: The posterior probability vector of pattern attribution ; core_tensors: A dictionary where the keys are pattern IDs and the values ​​are pointers to the corresponding core tensors. A pointer to the storage location; As a preferred approach, a probability threshold can be set. (For example ). For those modes whose mode attribution probability is lower than this threshold, the corresponding core tensor can not be calculated or stored, thereby further saving computing resources and storage space.

[0061] The above composite representation not only greatly compresses the data, but also decouples the structural characteristics of the tensor into two levels, i.e., the mode interpretation quantified by mode_probabilities and the specific performance under the corresponding mode represented by core_tensors, providing unprecedented convenience for subsequent deep analysis and efficient retrieval. For example, analysts can directly query all attention tensors that perform as "syntactic dependency" modes, and the system only needs to retrieve the entry with the highest mode probability in mode_probabilities, greatly improving the analysis efficiency.

[0062] An embodiment of a tensor database construction system of a large model: On the other hand, the present application also provides a tensor database construction system of a large model. As shown in Figure 5 A tensor database construction system of a large model includes a processor and a memory, and the memory stores computer program instructions, which, when executed by the processor, implement the tensor database construction method of a large model according to the first aspect of the present application.

[0063] The tensor database construction system of a large model also includes a communication interface and other components well known to those skilled in the art, the settings and functions of which are known in the art, and therefore will not be described here.

[0064] In this description, the term "computer readable medium" can be replaced by terms such as memory, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), electrically programmable read-only memory (EPROM), electrically erasable and programmable read-only memory (EEPROM), static random access memory (SRAM), erasable programmable logic and other similar devices. When the system reads certain instructions from the computer-readable medium, the instructions become part of the operating system or application program. In the context of this document, the terms "computer readable medium" and "computer readable storage medium" are used to generally refer to media such as removable storage drive, a hard disk installed in the computer, a magnetic disk that stores data, or any other appropriate medium that stores the desired information.

Claims

1. A method for constructing a tensor database of a large model, characterized in that, The method comprises: obtaining a plurality of intermediate layer tensors generated by a large language model in an inference process, and context meta-information corresponding to the plurality of intermediate layer tensors; dividing the plurality of intermediate layer tensors into different tensor clusters based on the context meta-information; for any tensor cluster, extracting a structural feature vector along at least one dimension of the tensors contained therein, the structural feature vector representing the covariance structure of the corresponding tensor in the dimension; using a Gaussian mixture model to cluster the set of structural feature vectors within the tensor cluster, obtaining a plurality of structural sub-patterns; constructing a dedicated shared basis for each structural sub-pattern, including: calculating a probability-weighted average value of the covariance matrix of the tensors corresponding to the structural sub-pattern, and performing eigenvalue decomposition on the obtained weighted average covariance matrix to obtain the shared basis; in response to receiving a new intermediate layer tensor, projecting it onto the plurality of shared bases corresponding to the tensor cluster to which it belongs, obtaining a set of core tensors; and combining the core tensors with the attribution probability information of the structural sub-patterns to obtain a composite core representation of the new intermediate layer tensor. 2.The method of claim 1, wherein, The division into different tensor clusters includes: intermediate layer tensors with the same model source, hierarchical index and tensor type, and whose input text semantic vectors have a similarity higher than a preset threshold, are divided into the same tensor cluster. 3.The method of claim 1, wherein, The extraction of the structural feature vector includes: expanding the tensor along the dimension to obtain a two-dimensional matrix; calculating the covariance matrix of the two-dimensional matrix; vectorizing the covariance matrix. 4.The method of claim 1, wherein, Using the Gaussian mixture model to cluster further includes: Within a preset range, the number of structural sub-patterns is automatically determined using the Bayesian information criterion or the Akaike information criterion. 5.The method of claim 1, wherein, The probability-weighted average value is obtained by weighting the posterior probability of each tensor belonging to the corresponding structural sub-pattern; The posterior probability is calculated based on the Gaussian mixture model. 6.The method of claim 1, wherein, The shared basis is constituted by eigenvectors corresponding to the first maximal eigenvalues of the weighted average covariance matrix, wherein is a preset decomposition rank.

7. The tensor database construction method of a large model according to claim 6, characterized in that, The decomposition rank is such that the energy of the retained eigenvalues accounts for more than 99% of the total energy. 8.The method of claim 1, wherein, When the attribution probability of a structural sub-pattern is lower than a preset probability threshold, the calculation or storage of the core tensor corresponding to the structural sub-pattern is stopped. 9.The method of claim 1, wherein, The composite core representation is a structured object, including: the identifier of the new intermediate layer tensor, the identifier of the tensor cluster to which it belongs, the attribution probability information of the structural sub-pattern, and a pointer to the storage location of a set of core tensors. 10.A tensor database construction system of a large model, characterized in that, A processor and a memory, the memory storing computer program instructions, when the computer program instructions are executed by the processor, implementing the tensor database construction method of the large model of any one of claims 1-9.

Images