Tensor database construction method and system of large model

Abstract

The application relates to the field of data processing, in particular to a tensor database construction method and system of a large model. The method captures tensors and clusters them according to semantics, then uses a Gaussian mixture model (GMM) to mine sub-patterns in the clusters and construct a shared basis. In online processing, a new tensor is decomposed into a composite representation composed of a mode probability and a core tensor through multi-basis projection. The representation retains structural information while greatly compressing data, realizing efficient structured query and analysis. The application mines multiple structural sub-patterns in the tensor clusters through the Gaussian mixture model, constructs a shared basis for multi-basis projection, generates a composite core representation to realize compression and analysis.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, in particular to a tensor database construction method and system for large models. BACKGROUND

[0002] With the rapid development of large language models (LLMs), understanding their internal mechanisms and explaining their behavior patterns have become key research focuses in the field of artificial intelligence. During the inference process of LLMs, intermediate layer tensors such as attention weights and hidden states are generated. By deeply studying the structure of these tensors, the internal computing patterns that the model relies on and can reuse when processing different tasks can be revealed, providing a theoretical basis for the explainability, compression, and optimization of the model.

[0003] In the prior art, a common analysis method is to first collect the intermediate layer tensors generated by the model when processing a batch of semantically similar inputs, and group them into a cluster as a behavior sample set of the model for a specific task. How to effectively mine one or more representative structural patterns from these complex high-dimensional tensor clusters. A feasible technical means is to extract a feature vector that can represent the internal structure of each tensor, and then apply an unsupervised clustering algorithm to the set of feature vectors to discover different structural patterns.

[0004] Among them, Gaussian Mixture Model (GMM) is one of the commonly used algorithms, because it can model data generated by multiple different distributions. However, in the context of LLM tensor analysis, direct application of GMM has inherent limitations. First, the cluster center (mean vector) learned by GMM can only represent the average form of one structural pattern, and it is easily affected by noise and outliers within the cluster, resulting in a less typical pattern represented. Secondly, the mean vector itself is an abstract statistical quantity and cannot directly provide a structured basis for the pattern that is simple, efficient and can be used for decomposition and reconstruction, making it difficult to apply the mined pattern to subsequent tasks such as tensor compression, efficient representation, or deep analysis. SUMMARY

[0005] In view of the difficulty of applying the cluster center of the above GMM to tensor compression, efficient representation or deep analysis problem, in the first aspect, the application proposes a tensor database construction method of large model, comprising: obtaining a plurality of intermediate layer tensors generated by a large language model in an inference process, and context meta information corresponding to the plurality of intermediate layer tensors; dividing the plurality of intermediate layer tensors into different tensor clusters based on the context meta information; for any tensor cluster, extracting a structural feature vector along at least one dimension of the tensor contained therein, the structural feature vector representing the covariance structure of the corresponding tensor in the dimension; using a Gaussian mixture model to cluster the structural feature vector set in the tensor cluster to obtain a plurality of structural sub-patterns; constructing a dedicated shared basis for each structural sub-pattern, comprising: calculating the probability weighted average of the covariance matrix of the tensor corresponding to the structural sub-pattern, and performing eigenvalue decomposition on the obtained weighted average covariance matrix to obtain the shared basis; in response to receiving a new intermediate layer tensor, projecting it onto the plurality of shared bases corresponding to the tensor cluster to which it belongs to obtain a group of core tensors; and combining the core tensors with the attribution probability information of the structural sub-patterns to obtain a composite core representation of the new intermediate layer tensor.

[0006] Unlike the prior art which usually directly stores the original intermediate layer tensors of the large language model or uses a single global compression method (such as PCA), the application proposes a new structured analysis and compression framework. By first clustering by semantics, then using a Gaussian mixture model to mine structural sub-patterns in the cluster and constructing a dedicated shared basis for each pattern, the tensor is finally decomposed into the attribution probability of the pattern and the composite representation of the core tensor. This method not only greatly compresses the storage volume of the tensor and saves hardware costs, but more importantly, it explicitly quantifies the implicit and difficult-to-use structural information in the tensor into discrete pattern attribution, making it possible for subsequent efficient structured query and deep analysis, solving the problem that traditional methods cannot balance high compression ratio and structural information preservation.

[0007] Further, the division into different tensor clusters comprises: dividing the intermediate layer tensors having the same model source, level index and tensor type, and the semantic vector similarity of the input text being higher than a preset threshold, into the same tensor cluster.

[0008] Compared with mixed analysis of all tensors or division only by level, the present application pre-clusters by model source, level, type and input text semantic similarity, ensuring that the subsequent structural analysis is performed on a highly similar function and context tensor set. This refined division avoids the noise and bias brought by mixed analysis of heterogeneous data, significantly improves the accuracy and purity of the Gaussian mixture model mining structural sub-patterns, makes the shared basis constructed for each cluster more representative, and thus improves the overall compression effect and analysis accuracy.

[0009] Further, the extracting the structural feature vector comprises: expanding the tensor along the dimension to obtain a two-dimensional matrix; calculating a covariance matrix of the two-dimensional matrix; and vectorizing the covariance matrix.

[0010] The application discloses that the structural feature vector is extracted by calculating the covariance matrix. Compared with directly analyzing original tensor values, the covariance matrix can more essentially and stably represent internal data correlation and distribution structure of the tensor in a specific dimension. This makes clustering of the structural mode no longer disturbed by specific value size, but focuses on inherent statistical characteristics, so that common structural sub-modes can be more accurately mined, and universality and robustness of the method to tensors with different value ranges are enhanced.

[0011] Further, the clustering by using the Gaussian mixture model further comprises: automatically determining the number of the structural sub-modes by using Bayesian information criterion or Akaike information criterion within a preset range.

[0012] Compared with a traditional method which needs to manually preset a clustering number (namely, K value) according to experience, the application automatically determines the number of the structural sub-modes by using Bayesian information criterion (BIC) or Akaike information criterion (AIC). This avoids subjectivity and tediousness of manual parameter adjustment, enables the model to adaptively find an optimal mode division according to complexity of data itself, and thus improves automation degree, objectivity and scalability of the whole analysis process.

[0013] Further, the probability weighted average value is obtained by weighting posterior probabilities of tensors belonging to corresponding structural sub-modes; and the posterior probabilities are calculated based on the Gaussian mixture model.

[0014] The application discloses a calculation mode of weighted average of the covariance matrix by using the posterior probability. This mode ensures that, when the shared basis is constructed, tensors which typically belong to the structural sub-mode can be more considered, and influence of abnormal or marginal tensors is reduced. Compared with simple arithmetic average, this weighted mode can construct a shared basis which is more pure and can represent core characteristics of the mode, so as to improve accuracy and compression efficiency of projection decomposition.

[0015] Further, the shared basis is composed of eigenvectors corresponding to the first M largest eigenvalues of the weighted average covariance matrix, wherein M is a preset decomposition rank. M is a preset decomposition rank.

[0016] Further, the decomposition rank makes energy of the reserved eigenvalue account for more than 99% of total energy.

[0017] ​Further, when the belonging probability of a structural sub-pattern is lower than a preset probability threshold, the calculation or storage of the core tensor corresponding to the structural sub-pattern is stopped.

[0018] Further, the composite core representation is a structured object, including an identifier of the new intermediate layer tensor, an identifier of the tensor cluster to which the new intermediate layer tensor belongs, belonging probability information of the structural sub-pattern, and a pointer to a storage location of a set of core tensors.

[0019] In a second aspect, the present application provides a tensor database construction system of a large model, comprising a processor and a memory, and the memory stores computer program instructions which, when executed by the processor, implement a tensor database construction method of a large model of the present application.

[0020] The technical effects of the present application are:

[0021] The present application proposes an innovative structured compression and analysis method aiming at the problem of large storage of intermediate layer tensors in large language models and difficulty in utilizing structural information. The core innovation is to extract the implicit and continuous structural features in the tensor into discrete and identifiable structural sub-patterns through semantic clustering and Gaussian mixture model (GMM) secondary mining. Then, by constructing a dedicated shared basis for each pattern and performing multi-basis projection, the original large tensor is decomposed into a composite representation including pattern belonging probability and multiple compact core tensors. This method not only realizes high-rate data compression, but also decouples and explicitly represents the structural information of the tensor for the first time, laying a foundation for efficient structured query and deep analysis. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a flow chart of a tensor database construction method of a large model according to an embodiment of the present application;

[0023] Figure 2 is a bar chart of tensor cluster size distribution according to an embodiment of the present application;

[0024] Figure 3 is a BIC curve diagram of GMM mode number selection according to an embodiment of the present application;

[0025] Figure 4 is a histogram of tensor reconstruction error distribution according to an embodiment of the present application;

[0026] Figure 5 is a structural block diagram of a tensor database construction system of a large model according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] With reference to the drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts are within the scope of the present application.

[0028] The specific implementation of the present application will be described in detail below with reference to the drawings.

[0029] A tensor database construction method for a large model embodiment:

[0030] As shown in Figure 1 , the tensor database construction method for a large model of the present application comprises:

[0031] S1, context-aware tensor capture and semantic-driven tensor cluster construction.

[0032] In this embodiment, first, the original intermediate layer tensor as an analysis object needs to be obtained from the inference process of a large language model (LLM), and context information is organized for it, laying a data foundation for subsequent pattern mining.

[0033] S1.1, non-intrusive tensor capture of LLM intermediate layers through framework hooks.

[0034] In an embodiment, by setting hooks (Hooks) to a large language model inference framework (such as PyTorch, TensorFlow), the intermediate layer tensors of the specified level can be captured non-intrusively when the model is forward propagated . For example, the attention weight tensor output by the self-attention module of a specific Transformer layer, or the hidden state tensor output by the feedforward network (FFN). The captured tensors are temporarily stored in a high-throughput temporary message queue (such as Kafka or RabbitMQ) and wait for subsequent batch processing.

[0035] S1.2, binding structured context meta-information containing semantic vectors to tensors.

[0036] While capturing tensors , the system captures and binds the complete context meta-information of the tensors. This meta-information is a structured JSON object used to describe tensor information, and at least contains the following fields:

[0037] tensor_id: a globally unique tensor identifier;

[0038] model_id: the model name and version that produces the tensor, e.g., "gpt-4";

[0039] layer_index: the layer index where the tensor originates, e.g., "encoder.layer.10";

[0040] tensor_type: the type of tensor, e.g., "attention_weights" or "hidden_state";

[0041] head_index: when tensor_type is attention_weights, this field identifies the specific attention head index;

[0042] input_id: a unique identifier for the input data that triggered the generation of the tensor;

[0043] input_text: the original input text;

[0044] input_semantic_vector: a semantic vector representation of the input text. This vector can be computed in real-time by a separate, lightweight sentence encoding model (e.g., Sentence-BERT) or retrieved from a pre-computed vector library.

[0045] S1.3, based on the semantic vector cosine similarity threshold, dynamically divide the tensors into clusters.

[0046] The background batch processing service consumes tensor data with meta information from the message queue. Based on the meta information, the tensors are dynamically divided into different tensor clusters . The grouping rules are configurable. As a typical grouping rule, tensors from the same model_id, the same layer_index, the same tensor_type, and whose input_semantic_vector has a cosine similarity in the vector space higher than the pre-set threshold are divided into the same cluster.

[0047] In this embodiment, the pre-set threshold is preferably 0.85-0.95. If the threshold is lower than 0.85, it may group tensors corresponding to inputs with large semantic differences into a cluster, resulting in a too mixed cluster structure pattern, which is not conducive to subsequent purification of the shared base. If the threshold is higher than 0.95, the grouping condition is too strict, which may result in a too small cluster size, and the structure pattern with generalization ability cannot be effectively learned. Therefore, controlling the threshold around 0.9 can ensure the consistency of the semantic consistency within the cluster while ensuring that there are enough data samples for pattern mining. Each tensor cluster A set of behavior samples generated by the model in a specific location when handling a similar task.

[0048] As Figure 2 shown, the five largest tensor clusters and the number of tensors contained in each cluster are shown after grouping by hierarchy and semantic similarity. For example, cluster_000 from the 10th layer contains about 120 tensors, which is the largest cluster. This figure intuitively demonstrates the effectiveness of the grouping mechanism, i.e., the ability to aggregate a large number of discrete tensors into behavior sample sets of different sizes for subsequent analysis.

[0049] S2, applying a Gaussian mixture model to mine multi-structure patterns in the cluster and generate a pattern-specific shared basis.

[0050] This step aims to address the limitations of the unimodal distribution assumption, and is responsible for automatically mining multiple discrete structure sub-patterns that may coexist from each tensor cluster , and constructing a dedicated, highly purified shared basis vector for each sub-pattern.

[0051] S2.1, calculate and vectorize the covariance matrix of the tensor along each dimension to extract structural features.

[0052] For any tensor cluster obtained in step S1 , and the tensors contained therein , for the dimension of each tensor, a feature vector that can represent its structural characteristics in that dimension needs to be extracted. For a three-order tensor, the dimension index .

[0053] First, calculate the matrix obtained after unfolding the tensor along the dimension ; then calculate the covariance matrix of the matrix, which can capture the variability and correlation of data in different directions, thus reflecting the internal structure of the tensor, specifically:

[0054] ;

[0055] where denotes the covariance matrix of the tensor in the dimension; denotes the transpose of the matrix . This covariance matrix contains the complete second-order structure information of the tensor in the dimension.

[0056] To facilitate subsequent cluster analysis, this high-dimensional symmetric covariance matrix needs to be... Vectorization yields a one-dimensional structural feature vector. Specifically, its upper triangular (or lower triangular) portion can be taken and flattened by row (or column) to form a one-dimensional vector.

[0057] S2.2 Utilize GMM to perform unsupervised clustering of structural features and adaptively determine the number of patterns.

[0058] Collect all tensors within the cluster in dimension The set of structural feature vectors on In this embodiment, to better verify the effectiveness of the subsequently constructed database, it is assumed that the vector set does not follow a single distribution, but is a mixture of multiple potential, discrete structural sub-patterns. Therefore, a Gaussian mixture model (Gaussian Mixture Model) is applied. , Perform unsupervised clustering on the vector set.

[0059] GMM is able to fit data because... The model is generated by mixing different Gaussian distributions. In this scenario, each Gaussian component ( ) precisely corresponds to a potential, discrete structural sub-pattern ( By maximizing expectation ( The algorithm trains the GMM, and the model can automatically learn the mean of each Gaussian component. (Representative Model) (Central typical structure), covariance (Representative Model) (Internal structural variation range) and mixed weights (Representative Model) (Frequency of occurrence in the cluster).

[0060] Optimal number of clusters (i.e., the number of modes) can be set within a preset range (e.g.) The search was performed using the Bayesian information criterion (...). ) or Akaike Information Guidelines ( (This will be automatically determined.) When the value is too small, meaningful sub-patterns cannot be distinguished, leading to underfitting of the model; when... When the value is too large, noise may also be fitted as an independent pattern, leading to model overfitting. The BIC / AIC criterion can effectively balance model complexity and goodness of fit, achieving adaptive discovery of the number of patterns. The training process of the GMM described above is a well-known technique, and its specific implementation will not be elaborated here.

[0061] After GMM training, for any one tensor in the cluster , the model can calculate its structure eigenvector belongs to any one of the different structure sub-patterns with posterior probability This probability vector provides soft information about the structure belonging of the tensor .

[0062] As shown in Figure 3 , it uses Bayesian Information Criterion (BIC) to adaptively find the optimal number of modes (K) . In the figure, the X-axis is the number of components (i.e. the number of modes) that GMM tries, and the Y-axis is the corresponding BIC value. According to the principle of selecting the model that minimizes the BIC value, when the number of components is 4, the BIC value reaches the lowest point, that is, for the current analyzed tensor cluster, the system automatically discovers that there are 4 main structure sub-patterns coexisting inside it.

[0063] S2.3, construct a weighted covariance matrix based on the GMM posterior probability, and decompose to generate the shared basis of each mode.

[0064] For each structure sub-pattern discovered by GMM , construct its own dedicated, highly purified shared basis. Specifically, calculate the probability-weighted average covariance matrix of mode in dimension . This formula can make tensors that are highly consistent with the structure of mode contribute more when constructing the representative covariance matrix of this mode by using the posterior probability as the weight, while the contribution of inconsistent tensors is suppressed, specifically:

[0065] ;

[0066] where represents the posterior probability of tensor belonging to mode ; represents the covariance matrix of tensor ; represents the total number of tensors in the cluster; represents a very small positive number used to avoid the denominator being zero, which can be taken as an empirical value of 1e-6 in this embodiment.

[0067] This finally generated can accurately depict the unique structure of mode , avoiding pollution by tensors of other modes. When the structure characteristics of a tensor are With the mode center The closer, the greater its posterior probability value, the greater its proportion in the weighted average, thus dominating the formation.

[0068] Finally, the weighted average covariance matrix of each mode is respectively decomposed into eigenvalues . Select the eigenvectors corresponding to the first maximal eigenvalues to form the shared factor matrix (i.e., shared basis) of the mode in this dimension, where is the size of the dimension , and is the rank of the decomposition.

[0069] In this embodiment, the rank of the decomposition affects the balance between compression rate and fidelity. Preferably, the value of can be set so that the retained eigenvalue energy accounts for or more of the total energy, or it can be taken as a tunable hyperparameter, for example, with a value of or .

[0070] As shown in Figure 4 , this histogram shows the distribution of relative errors between the original tensor and the reconstructed tensor after the tensor is compressed by multi-base projection and then reconstructed using the core tensor and shared basis. The chart shows that the reconstruction errors of most tensors are concentrated between 0.5 and 0.6, with an average error of 0.4536. This value is a key indicator of information fidelity, and its size can be controlled by adjusting hyperparameters such as the rank Rn mentioned in S2.3, thereby balancing compression efficiency and data accuracy.

[0071] S3, construct a hybrid storage database to store the tensor clusters, structural modes, and shared bases in a structured manner.

[0072] This step is responsible for the structured and queryable storage of the shared factor matrices calculated in step S2, forming the knowledge base of the entire system. The database uses a hybrid storage scheme, for example, using a document database (such as MongoDB) to store metadata and model information, and using object storage (such as "S3 path") or high-performance key-value storage (such as RocksDB) to store numerical data.

[0073] The data model is designed as follows:

[0074] TensorClusters collection: stores the definition information of tensor clusters;

[0075] cluster_id: unique identifier of the cluster;

[0076] model_id, layer_index, tensor_type: meta information defining the cluster;

[0077] semantic_centroid: mean of all input_semantic_vector in the cluster, representing the semantic center of the cluster;

[0078] gmm_model_path_dim_n: the storage path (e.g. "S3 path") of the trained GMM model for this cluster dimension ;

[0079] StructureModes collection: stores the information of discovered structural sub-mode in each cluster;

[0080] mode_id: unique identifier of the mode;

[0081] cluster_id: the cluster ID it belongs to;

[0082] dimension: the corresponding dimension index ;

[0083] gmm_component_index: the corresponding GMM component index;

[0084] mixing_weight: the frequency of the mode appearing in the cluster (i.e. the mixing weight of the GMM );

[0085] shared_basis_id: the ID pointing to the mode-specific shared basis;

[0086] SharedBases storage: stores the specific shared factor matrices (basis vectors) in key-value pairs;

[0087] Key: shared basis ID, e.g. "basis_pqr";

[0088] Value: serialized NumPy array or HDF5 file, containing the shared factor matrix set of the mode across all dimensions ;

[0089] S4, Real-time decomposition and compressed storage of new tensors based on multi-basis projection and composite representation.

[0090] This step is the online service part of the system, responsible for real-time and efficient decomposition and storage of newly generated tensors.

[0091] S4.1. Locate the cluster to which the new tensor element belongs based on the new tensor element information, and load the GMM model to determine its pattern affiliation probability.

[0092] When the online service receives a new tensor and its metadata At that time, firstly according to Locate the tensor cluster `cluster_abc` to which the given tensor belongs in the `TensorClusters` collection; then, load the corresponding GMM model for each dimension (e.g., `gmm_abc_d1.pkl`); next, ... Real-time extraction of structural feature vectors in various dimensions The feature vector is then input into the corresponding GMM model to calculate... The posterior probability vectors belonging to each structural sub-pattern within this cluster .

[0093] S4.2. Project in parallel to all mode bases to generate a composite representation containing probability and core tensor and store it in the library.

[0094] According to the output of GMM, the service retrieves all data from StructureModes and SharedBases at once. The set of shared factor matrices corresponding to each pattern. Execute separately The parallel projective decomposition involves projecting the decomposition onto the shared base space of each pattern, resulting in a set of... One core tensor . No. The calculation of each core tensor is based on the inverse process of Tucker decomposition, specifically:

[0095] ;

[0096] in express In mode The core tensor under the base; Indicates along the first Matrix product of modulo; Representation pattern In dimensions Transpose of the shared base on.

[0097] The system does not store large raw tensors Instead, it generates and stores a highly condensed composite core representation ( This representation is a structured object containing the following information:

[0098] entry_id: The unique ID of this stored entry;

[0099] original_tensor_id: ID of the original tensor ;

[0100] cluster_id: ID of the cluster to which it belongs

[0101] mode_probabilities: posterior probability vector of the mode assignment ;

[0102] core_tensors: a dictionary with mode IDs as keys and pointers to the storage locations of the corresponding core tensors as values ;

[0103] As a preferred solution, a probability threshold (e.g. ) can be set. For those modes whose mode assignment probability is lower than the threshold, their corresponding core tensors can not be calculated or stored, thus further saving computing resources and storage space.

[0104] The above composite representation not only greatly compresses the data, but also decouples the structural characteristics of the tensor into two aspects: the mode interpretation quantified by mode_probabilities and the specific manifestation under the corresponding mode represented by core_tensors, providing unprecedented convenience for subsequent deep analysis and efficient retrieval. For example, analysts can directly query all attention tensors that manifest as the “syntactic dependency” mode, and the system only needs to retrieve the entry with the highest mode probability in mode_probabilities, greatly improving analysis efficiency.

[0105] An embodiment of a tensor database construction system of a large model:

[0106] In another aspect, the present application also provides a tensor database construction system of a large model. As shown in the figure, a tensor database construction system of a large model includes a processor and a memory, and the memory stores computer program instructions, which, when executed by the processor, implement a tensor database construction method of a large model according to the first aspect of the present application. Figure 5

[0107] The tensor database construction system of a large model also includes a communication interface and other components familiar to those skilled in the art, the settings and functions of which are known in the art, and therefore will not be described here.

[0108] ​​In this description, the term "computer readable medium" can be replaced by terms such as memory, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), electrically programmable read-only memory (EPROM), electrically erasable and programmable read-only memory (EEPROM), static random access memory (SRAM), erasable programmable logic and other similar devices. When the system reads certain instructions from the computer-readable medium, the instructions become part of the operating system or application program. In the context of this document, the terms "computer readable medium" and "computer readable storage medium" are used to generally refer to media such as removable storage drive, a hard disk installed in the computer, a magnetic disk that stores data, or any other appropriate medium that stores the desired information.

Claims

1. A method for constructing a tensor database of a large model, characterized in that, The method comprises: obtaining a plurality of intermediate layer tensors generated by a large language model in an inference process, and context meta-information corresponding to the plurality of intermediate layer tensors; dividing the plurality of intermediate layer tensors into different tensor clusters based on the context meta-information; for any tensor cluster, extracting a structural feature vector along at least one dimension of the tensors contained in the tensor cluster, the structural feature vector representing the covariance structure of the corresponding tensor in the dimension; using a Gaussian mixture model to cluster the set of structural feature vectors in the tensor cluster, obtaining a plurality of structural sub-patterns; constructing a dedicated shared basis for each structural sub-pattern, including: calculating a probability weighted average of the covariance matrix of the tensors corresponding to the structural sub-pattern, and performing eigenvalue decomposition on the obtained weighted average covariance matrix to obtain the shared basis; in response to receiving a new intermediate layer tensor, projecting it onto the plurality of shared bases corresponding to the tensor cluster to which it belongs, obtaining a set of core tensors; and combining the core tensors with the attribution probability information of the structural sub-patterns to obtain a composite core representation of the new intermediate layer tensor, including: when an online service receives a new tensor and its meta-information, finding the tensor cluster to which it belongs according to the meta-information; loading the dimension mixture model corresponding to the cluster; extracting the structural feature vector of each dimension of the new tensor in real time, and inputting the feature vector into the corresponding mixture model to calculate the posterior probability vector of the new tensor belonging to each structural sub-pattern in the cluster; According to the output of the mixed model, all shared factor matrix sets corresponding to the modes are obtained at one time; the new tensor is respectively subjected to times parallel projection-type decompositions, which are projected onto the shared basis space of each mode, and a group of core tensors is calculated according to the inverse process of Tucker decomposition. 2.The method of claim 1, wherein, the division into different tensor clusters includes: dividing intermediate layer tensors with the same model source, hierarchical index and tensor type, and whose semantic vectors of input text are more similar than a preset threshold, into the same tensor cluster. 3.The method of claim 1, wherein, the extraction of the structural feature vector includes: expanding the tensor along the dimension to obtain a two-dimensional matrix; calculating the covariance matrix of the two-dimensional matrix; vectorizing the covariance matrix. 4.The method of claim 1, wherein, using the Gaussian mixture model for clustering further includes: within a preset range, using the Bayesian information criterion or the Akaike information criterion to automatically determine the number of structural sub-patterns. 5.The method of claim 1, wherein, the probability weighted average is obtained by weighting the posterior probability of each tensor belonging to the corresponding structural sub-pattern; the posterior probability is calculated based on the Gaussian mixture model. 6.The method of claim 1, wherein, The shared basis is constituted by eigenvectors corresponding to the first maximal eigenvalues of the weighted average covariance matrix, wherein is a preset decomposition rank.

7. The tensor database construction method of a large model according to claim 6, characterized in that, the decomposition rank is such that the energy of the retained eigenvalues accounts for more than 99% of the total energy. 8.The method of claim 1, wherein, when the attribution probability of a structural sub-pattern is lower than a preset probability threshold, stop calculating or storing the core tensor corresponding to the structural sub-pattern. 9.The method of claim 1, wherein, the composite core representation is a structured object, including: the identifier of the new intermediate layer tensor, the identifier of the tensor cluster to which it belongs, the attribution probability information of the structural sub-pattern, and a pointer to the storage location of a set of core tensors. 10.A tensor database construction system of a large model, characterized in that, including a processor and a memory, the memory storing computer program instructions, when the computer program instructions are executed by the processor, implementing the tensor database construction method of the large model in any one of claims 1-9.

Images