Model compression storage method, model decompression method, electronic device and medium

Abstract

The present application discloses a model compression storage method, a model decompression method, an electronic device and a medium, and relates to the field of data storage technology, including: splitting the weight matrix of the target model into a combination of multiple target vectors, performing a reorganization operation on the combination of multiple target vectors, and then generating a vector set based on the reorganized vectors. The reorganization process can optimize and integrate the target vectors, and the generated vector set can more efficiently represent the original weight matrix. The compressed weight matrix is ​​stored based on the reorganized vectors and the vector set, thereby fully mining the structural information inside the weight matrix, avoiding the direct storage of the entire weight matrix, and thus reducing storage space. Therefore, the problem of model storage occupying a large amount of storage space in the related art can be solved.

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and in particular to a model compression storage method, a model decompression method, an electronic device, and a medium. Background Art

[0002] With the rapid development of artificial intelligence (AI), the Transformer architecture model, with its superior performance, has been widely adopted in fields such as natural language processing and computer vision, and holds great promise. However, as models continue to expand in size and application scenarios become increasingly complex, storage requirements have become a key obstacle to their further development and application. Traditional model storage methods typically require a large amount of storage space, placing stringent demands on the capacity and performance of storage devices.

[0003] Related technologies propose low-rank decomposition of the model's linear layer parameters and joint quantization of activations and weights, using low-bit representation to reduce the number of parameters; the intermediate activations after low-rank factor calculation are also quantized to low bits, achieving hardware-friendly model low-rank quantization acceleration, improving prediction and inference speed and reducing storage requirements.

[0004] However, the focus of related technologies is on low-rank quantization, model acceleration, and accuracy recovery. Reducing storage requirements mainly relies on quantization and low-rank decomposition. However, in the Transformer architecture model, there are a large number of parameter matrices, which occupy a huge amount of storage space. Summary of the Invention

[0005] The present application provides a model compression storage method, a model decompression method, an electronic device and a medium, so as to at least solve the problem in the related art that model storage occupies a large amount of storage space.

[0006] The present application provides a model compression storage method, comprising: decomposing a weight matrix of a target model into a combination of multiple target vectors; recombining the combination of multiple target vectors and generating a vector set based on the recombined vectors; and storing the compressed weight matrix based on the recombined vectors and the vector set.

[0007] The present application provides a model decompression method, comprising: obtaining a compressed target model, wherein the compressed target model is compressed using the model compression storage method of the above-mentioned embodiment; identifying the compressed weight matrix in the compressed target model; and decompressing the compressed weight matrix.

[0008] The present application also provides a model compression storage device, including: a decomposition module for decomposing the weight matrix of the target model into a combination of multiple target vectors; a generation module for recombining the combination of multiple target vectors and generating a vector set based on the recombined vectors; and a storage module for storing the compressed weight matrix based on the recombined vectors and the vector set.

[0009] The present application also provides a model decompression device, including: an acquisition module for acquiring a compressed target model, wherein the compressed target model is compressed using the model compression storage device of the above-mentioned embodiment; an identification module for identifying the compressed weight matrix in the compressed target model; and a decompression module for decompressing the compressed weight matrix.

[0010] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing any one of the above-mentioned model compression storage methods and model decompression method steps when executing the computer program.

[0011] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, any one of the above-mentioned model compression storage methods and model decompression method steps is implemented.

[0012] The present application also provides a computer program product, including a computer program, which implements any of the above-mentioned model compression storage methods and model decompression method steps when executed by a processor.

[0013] Through this application, the weight matrix of the target model is split into a combination of multiple target vectors, the combination of multiple target vectors is reorganized, and then a vector set is generated based on the reorganized vectors. The reorganization process can optimize and integrate the target vectors, and the generated vector set can more efficiently represent the original weight matrix. The compressed weight matrix is ​​stored based on the reorganized vectors and the vector set. Thus, the structural information inside the weight matrix can be fully mined, avoiding the direct storage of the entire weight matrix, thereby reducing storage space. Therefore, the problem of model storage occupying a large amount of storage space in the related art can be solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0015] Figure 1 An example diagram of the execution of a query embedding generation module provided in one embodiment of the present application;

[0016] Figure 2 An example diagram of the execution of a key embedding generation module provided in one embodiment of the present application;

[0017] Figure 3 An example diagram of executing a value embedding generation module provided in one embodiment of the present application;

[0018] Figure 4 An example diagram of sharing the same input embedding execution provided by one embodiment of the present application;

[0019] Figure 5 An example diagram of a feedforward neural network layer execution provided for one embodiment of the present application;

[0020] Figure 6 A flowchart of the model compression storage method provided in an embodiment of the present application;

[0021] Figure 7 A flowchart of the model decompression method provided in an embodiment of the present application;

[0022] Figure 8 A block diagram of a model compression storage device provided in an embodiment of the present application;

[0023] Figure 9 A block diagram of a model decompression device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0025] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0026] As deep learning continues to advance, the size and complexity of models are increasing. Traditional model storage methods face significant challenges. These methods require significant storage space, placing high demands on storage device capacity and performance. This also limits the application and deployment of models in resource-constrained environments. To address this challenge, low-bit quantization methods have emerged. The core of low-bit quantization lies in processing numerical values ​​within the model. In deep learning models, parameters are typically stored as 32-bit floating-point numbers. While this provides high accuracy, it also consumes significant storage space. Low-bit quantization technology quantizes these values ​​from 32-bit floating-point numbers to 8 bits or even lower. Technically, low-bit quantization is achieved by discretizing the original numerical values. For example, 8-bit quantization divides the originally continuous numerical range into 256 discrete levels, each represented by an 8-bit binary number. This allows each parameter to be stored in only 8 bits of space, significantly reducing storage space compared to 32-bit floating-point numbers.

[0027] On the other hand, from a structural perspective, weight matrices often exhibit specific regularities and patterns. In the weight matrices of the convolutional layers of convolutional neural networks, different convolution kernels may have similar structures, or certain rows and columns may have significant linear correlations. These structural features provide an opportunity for the application of low-order matrix representation techniques. Therefore, low-order matrix representation has become an important means of reducing storage requirements. Using low-order matrix representation techniques, the originally high-dimensional and complex weight matrix can be decomposed into a combination of low-order matrices. Therefore, after completing the low-order matrix decomposition of the weight matrix, only the relevant information of these low-order matrices needs to be stored during storage, significantly reducing storage overhead compared to directly storing the original weight matrix.

[0028] The embodiments of the present application offer a fresh perspective. Models based on the Transformer architecture have complex internal structures and numerous parameters. The vast number of parameter matrices creates a storage burden. Numerous multi-head attention mechanisms and feedforward neural network layers involve multiple parameter matrices at each layer. These parameter matrices can easily span tens of thousands of rows and columns. Using traditional storage methods, storing these matrices in their entirety requires a significant amount of storage space. Therefore, the embodiments of the present application decompose these parameter matrices by row or column, representing them as a combination of several vectors (row or column vectors). This in-depth analysis and reorganization of the matrix structure provides a new foundation for subsequent storage optimization. Secondly, a vector set is generated based on the reorganized vectors. This reorganization process optimizes and integrates the target vectors. The resulting vector set more efficiently represents the original weight matrix. Storing the compressed weight matrix based on the reorganized vectors and vector set avoids directly storing the complete matrix, significantly reducing storage requirements. This can significantly save storage resources, especially when processing large-scale Transformer model weights.

[0029] Furthermore, if these vectors can be represented as linear combinations of basis vectors, that is, a vector can be represented as a combination of a set of basis vectors and a set of coefficients, then each vector can be expressed using only one set of coefficients. Since the basis vectors are shared by all vectors, they can be stored separately, thus saving storage space.

[0030] To help those skilled in the art better understand the present invention, the following explains the Transformer architecture model involved in this application, including (1) a query embedding generation module; (2) a key embedding generation module; (3) a value embedding generation module; and (4) a feedforward neural network layer (also called a feedforward neural network module). The details are as follows:

[0031] (1) Description of the query embedding generation module

[0032] like Figure 1 As shown, the query embedding generation module embeds its module input ( ) is converted to a query embedding ( ). Assume that the module input embedding yes A column matrix with 1 row and 1 column, that is ; Query Embedding yes A column matrix with 1 row and 1 column, that is ; The query embedding generation module contains the weight matrix yes OK, The matrix of columns, that is Then, the process of module input embedding generating module output query embedding through query embedding can be formulated as: .

[0033] (2) Description of the key embedding generation module

[0034] like Figure 2 As shown, the key embedding generation module embeds its module input ( ) is converted to a key embedding ( ). Assume that the module input embedding yes A column matrix with 1 row and 1 column, that is ;Key embedding yes A column matrix with 1 row and 1 column, that is ;The key embedding generation module contains the weight matrix yes OK, The matrix of columns, that is Then, the process of module input embedding generating module output key embedding through key embedding can be formulated as: .

[0035] (3) Description of the value embedding generation module

[0036] like Figure 3 As shown, the value embedding generation module embeds its module input ( ) is converted to a value embedding ( ). Assume that the input embedding yes A column matrix with 1 row and 1 column, that is ; Value embedding yes A column matrix with 1 row and 1 column, that is ; The weight matrix included in the value embedding generation module yes OK, The matrix of columns, that is Then, the process of module input embedding generating module output value embedding through value embedding can be formulated as: .

[0037] Further, if Figure 4 As shown in Figure 2, within the Transformer-based model, the query embedding generation module, key embedding generation module, and value embedding generation module are always used as a whole, sharing the same input embedding ( When single-head attention is used, the input embedding ( ) are input into the query embedding generation module, key embedding generation module and value embedding generation module respectively; In addition, when multi-head attention is used, it contains multiple groups of query embedding generation modules, key embedding generation modules and value embedding generation modules, and the module input embedding ( ) are input into each set of query embedding generation module, key embedding generation module and value embedding generation module respectively.

[0038] It should be noted that the input embedding (X) of the above module can be the word embedding generated by the word embedding encoder when the word is input into the model, or it can be the embedding within the model. Taking text input as an example, when the text is input into the model, the text is divided into several words, and then each word corresponds to a word embedding. When representing the above word embedding, then usually ;when When representing the internal embedding of the model, It should be noted that and When the input is an image, the image is divided into several image slices, and then the image slices correspond to a word element embedding.

[0039] (4) Feedforward neural network layer (weights are )

[0040] like Figure 5 As shown, the feedforward neural network layer embeds its input into ( ) is converted to an output embedding ( ). Assume that the input embedding yes A column matrix with 1 row and 1 column, that is ; Output embedding yes A column matrix with 1 row and 1 column, that is ; The weight matrix contained in the feedforward neural network layer yes OK, The matrix of columns, that is .So, In the embodiment of the present application, the feedforward neural network layer refers to the feedforward neural network layer inside each Transformer module, and its input embedding and output embedding The dimensions are the same, i.e. .

[0041] The present application will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0042] Specifically, Figure 6 A flowchart of a model compression storage method provided in an embodiment of the present application.

[0043] like Figure 6 As shown, the model compression storage method includes the following steps:

[0044] In step S101 , the weight matrix of the target model is decomposed into a combination of multiple target vectors.

[0045] The target model may include the query embedding generation module, the key embedding generation module, the value embedding generation module and the feedforward neural network layer, each module containing multiple weight matrices.

[0046] In some embodiments, decomposing a weight matrix of a target model into a combination of multiple target vectors includes: obtaining a target decomposition dimension of the target model; and decomposing the weight matrix into a combination of multiple target vectors according to the target decomposition dimension, wherein the target decomposition dimension includes at least one of a row dimension and a column dimension.

[0047] It is understandable that the embodiment of the present application can decompose these weight matrices according to the row dimension or column dimension so that they can be represented as a combination of several vectors (row vectors or column vectors). Furthermore, that is, a vector can be represented as a combination of a set of vector serial numbers and a set of coefficients, then each vector can be expressed by only one set of coefficients, thereby saving storage space. It should be noted that in the embodiment of the present application, the serial number represents the serial number of the vector in the vector set. The vector can be retrieved from the vector set through this serial number. Each row of data can be approximately restored based on the retrieved vector and coefficient.

[0048] Specifically, the target decomposition dimension determines how the matrix will be split, including row and column dimensions. The row dimension runs horizontally across the matrix, with each row representing a set of related data features. The column dimension runs vertically across the matrix, with each column also containing specific data information. When the target decomposition dimension is row, the weight matrix is ​​split into multiple row vectors. Each row vector contains the values ​​of the elements in each column for that row. If the target decomposition dimension is column, the weight matrix is ​​split into multiple column vectors. Each column vector contains the values ​​of the elements in each row for that column. Different decomposition methods result in different storage structures. For example, in a neural network's weight matrix, rows may correspond to different neurons, and columns may correspond to different input features. Row-based decomposition allows each row vector to be stored separately, facilitating the management of information for each neuron. Column-based decomposition is more suitable for storing and processing input feature information. Choosing the appropriate decomposition method can reduce storage redundancy and improve storage efficiency.

[0049] Furthermore, before decomposing the weight matrix of the target model into a combination of multiple target vectors, the method further includes: obtaining the number of weight matrices; if there are multiple weight matrices, grouping the multiple weight matrices according to the target decomposition dimension.

[0050] Understandably, in the Transformer architecture, weight matrices are widely distributed across various functional modules, such as the query embedding generation module, key embedding generation module, value embedding generation module, and feedforward neural network layers. The number of weight matrices in different modules is numerous, each with specific functions and properties. Accurately identifying these matrices and grouping them facilitates unified processing of matrices in the same group, enabling faster group reading and calculation, enhancing model performance, and providing strong support for model application in complex scenarios.

[0051] During the actual execution process, if there are multiple weight matrices, the embodiment of the present application can divide the weight matrices with the same dimension into a group on the target decomposition dimension, so that they can be operated in a unified manner in subsequent processing to improve computing efficiency.

[0052] For example, in a complex deep learning model, there may be multiple weight matrices, and different weight matrices correspond to different layers or different functional modules of the model. Obtaining the number of weight matrices can give us a preliminary understanding of the scale and complexity of the model. For example, in a multi-layer convolutional neural network, the convolution kernel of each layer corresponds to a weight matrix. By counting the number of weight matrices, the embodiment of the present application can know how many convolution layers the network has, thereby providing basic information for subsequent processing. In actual operation, the embodiment of the present application can obtain the number of weight matrices through the structural information of the model.

[0053] Specifically, the embedding generation module, key embedding generation module, value embedding generation module, and weight matrix in the feedforward neural network layer of the target model are grouped. The grouping basis of the embodiment of the present application can be the row dimension of the weight matrix, that is, matrices with the same row dimension are grouped into the same group.

[0054] Assume that all the above weight matrices can be divided into groups, where group i contains weight matrices, which have the same number of columns (i.e., the same row dimensions) and are , then the i-th group can be expressed as .

[0055] <1>

[0056] in, represents the set of weight matrices of group i, , , Respectively represent the number of rows of each weight matrix in group i, represents the number of columns of all weight matrices in group i.

[0057] Because each group of weight matrices uses the same compression method, in order to simplify the discussion, the embodiment of the present application can omit the superscript , a set of weight matrices is represented by the following formula.

[0058] <2>

[0059] In step S102, a combination of multiple target vectors is reorganized, and a vector set is generated based on the reorganized vectors.

[0060] It's understandable that after decomposing the weight matrix into multiple target vectors, while each contains specific information, these vectors are relatively scattered, making them difficult to manage and process. Recombining multiple target vectors aims to uncover the inherent connections and patterns between them, integrating this scattered information to form new, more representative and regular vectors—the recombined vectors. This helps reduce data redundancy, improves data representation efficiency, and lays the foundation for subsequent generation of vector sets and further optimization of model storage and computational efficiency.

[0061] Specifically, the embodiment of the present application can be a set of weight matrices Decompose into (in, )indivual A collection of n-dimensional row vectors , as shown in Formula 3 below:

[0062] <3>

[0063] in, represent The set of dimensional real row vectors, represent row vectors, represents the sequence number of the row vector, is a row vector in the set The index in .

[0064] In some embodiments, recombining a combination of multiple target vectors includes: setting a number threshold of central vectors according to the number of multiple target vectors; clustering the multiple target vectors according to the number threshold of central vectors; and using the central vector obtained by clustering as a recombined vector.

[0065] In the embodiments of the present application, clustering can compress a large number of target vectors into a small number of center vectors. This greatly reduces the amount of data stored and improves data storage efficiency. This data compression effect is particularly evident when processing large-scale data sets, significantly reducing storage costs. Clustering is the process of grouping similar target vectors into the same group. The number of center vectors determines the number of cluster groups, as follows:

[0066] The embodiment of the present application can use a clustering algorithm (such as K-means clustering algorithm) to indivual A set of dimensional vectors Processing is performed and the vectors are clustered into center vectors, where In the embodiments of this application , In the embodiment of the present application, the clustering obtained A vector is also called The reorganized vectors are then reassembled into basis vectors. Therefore, setting a threshold for the number of center vectors based on the number of target vectors allows for flexible control of clustering effectiveness, preventing overly dispersed or concentrated clustering results and ensuring that the reorganized vectors more accurately represent the characteristics of the original target vectors. Secondly, clustering multiple target vectors and using the resulting center vectors as the reorganized vectors significantly reduces data redundancy, allowing similar vectors to be represented by a small number of center vectors, thereby reducing storage requirements and improving storage efficiency.

[0067] In some embodiments, generating a vector set according to the reorganized vector includes: determining a sequence number of the reorganized vector in the vector set; and generating the vector set according to the reorganized vector and the sequence number of the reorganized vector.

[0068] It's understood that after decomposing the weight matrix into multiple target vectors and recombining them, a series of recombined vectors are generated. To effectively manage and utilize these recombined vectors, each recombined vector needs to be assigned a sequence number within the vector set. Sequence numbers can be assigned based on the order in which the vectors were generated, or they can be sorted based on certain characteristic attributes, without specific limitation.

[0069] Taking an array as an example, the process of generating a vector set is to store the reorganized vectors in the corresponding positions of the array in sequence according to their sequence numbers. For example, suppose there are three reorganized vectors b1, b2, and b3, with sequence numbers 1, 2, and 3, respectively. In the array, b1 will be stored in the first position, b2 in the second, and b3 in the third. When storing vectors, other relevant information such as the vector's dimension and data type can also be stored to facilitate subsequent processing.

[0070] The set of reorganized vectors is called a vector set (also called a basis vector set), which is expressed as . Each set of weight matrices corresponds to a set of vectors, then The group weight matrix corresponds to These vector sets will be stored in the storage unit as a "vector lookup table". During the model inference process, the basis vectors are read from the vector lookup table and the target vectors in the weight matrix are restored using the vectors.

[0071] In step S103 , the compressed weight matrix is ​​stored based on the reorganized vector and the vector set.

[0072] Understandably, the model's weight matrix is ​​massive, and storing it directly would take up a significant amount of space. Decomposing and reorganizing it into vector sets can effectively reduce storage requirements. After decomposing the weight matrix into vectors by row or column, a clustering algorithm is used to obtain the reorganized vectors, forming a vector set. The vectors in these vector sets are shareable. For example, a basis vector can be shared by multiple target vectors. This eliminates the need to store the entire weight matrix, requiring only the basis vectors and a small amount of coefficients and index information. This significantly reduces storage costs and makes it easy to deploy the model on devices with limited storage resources.

[0073] Therefore, the embodiment of the present application can express each column or each row of the weight matrix as a combination of several vectors, which can fully explore the structural information inside the weight matrix, avoid direct storage of the entire weight matrix, thereby reducing storage space, and effectively improving storage efficiency. It provides strong support for the application and promotion of the Transformer architecture model in various resource-constrained environments, and has important practical value.

[0074] In some embodiments, storing a compressed weight matrix based on a reorganized vector and a vector set includes: identifying a first identifier of the vector set; representing multiple target vectors using the reorganized vector; and storing the vector set of the compressed weight matrix, the first identifier, and a second identifier representing the multiple target vectors.

[0075] In this embodiment, the sequence number of a vector set can be obtained; using the sequence number as the first identifier of the vector set ensures the accuracy and consistency of the vector set during storage and use. Each vector set has a unique sequence number, which effectively avoids confusion or misreference caused by unclear identification.

[0076] The present application can disassemble the weight matrix by row or column and perform a reorganization operation through the above-mentioned embodiment. The embodiment of the present application can extract information that can uniquely identify the target vector, i.e., the second identifier, from the reorganized vector. And adopt a method of storing vector sets, sequence numbers of vector sets, and second identifiers representing multiple target vectors separately. Through this separate storage, not only the internal structure information of the weight matrix is ​​captured more accurately, but also the storage method is greatly optimized, avoiding direct storage of the entire weight matrix, thereby effectively reducing storage space.

[0077] In some embodiments, representing multiple target vectors based on a reorganized vector includes: calculating second identifiers for the multiple target vectors based on the reorganized vector. In some embodiments, calculating second identifiers for the multiple target vectors based on the reorganized vector includes: calculating the Euclidean distance between the target vector and the reorganized vector; determining the reorganized vector that matches the target vector based on the Euclidean distance; identifying the sequence number of the reorganized vector that matches the target vector; calculating the vector ratio between the target vector and the reorganized vector that matches the target vector; and generating second identifiers for the multiple target vectors based on the sequence number and vector ratio of the reorganized vector that matches the target vector. Thus, by calculating the Euclidean distance, determining the matching vector, identifying the sequence number, and calculating the vector ratio to generate the second identifier, accurate representation and efficient indexing of the target vector are achieved, which helps reduce storage requirements, improve data retrieval and model computation efficiency, and optimize the storage and runtime performance of the Transformer architecture model.

[0078] The embodiment of the present application can be further explained by taking the target vector as a row vector as an example. If each row vector of the parameter matrix in the Transformer architecture model can be expressed as a combination of several basis vectors and corresponding coefficients, then when performing storage operations, each row vector can be represented by its corresponding coefficient and the serial number of the basis vector. The effect of reducing storage space can be achieved if and only if the storage space occupied by the coefficient and the serial number of the basis vector corresponding to each row vector is less than the space occupied by the original row vector. In the most ideal extreme case, each row vector only needs to use one coefficient and one corresponding serial number of the basis vector to complete the expression.

[0079] For a set of row vectors Any row vector in , according to the Euclidean distance formula between vectors, we can get The closest basis vector to its Euclidean distance is calculated in , assuming that the reorganized vector is , then record its serial number .

[0080] Assume that the row vector , the nearest recombined vector is ,So , represents the Euclidean distance. Therefore, .

[0081] After the above steps, each vector Can be expressed as a and a coefficient Because each set of weight matrices shares a set of basis vectors, there is no need to Direct Recording , only record the corresponding serial number Here you can see each vector It can be expressed by two numbers and , significantly reducing the amount of storage.

[0082] According to the model compression storage method proposed in the embodiment of the present application, the weight matrix of the target model is split into a combination of multiple target vectors, the combination of multiple target vectors is reorganized, and then a vector set is generated based on the reorganized vectors. The reorganization process can optimize and integrate the target vectors, and the generated vector set can more efficiently represent the original weight matrix. The compressed weight matrix is ​​stored based on the reorganized vectors and the vector set. Thus, the structural information inside the weight matrix can be fully mined, avoiding the direct storage of the entire weight matrix, thereby reducing storage space. Therefore, the problem of model storage occupying a large amount of storage space in the related art can be solved.

[0083] Secondly, this application also provides a model decompression method, such as Figure 7 Shown, including:

[0084] In step S201 , a compressed target model is obtained, wherein the compressed target model is compressed using a model compression storage method.

[0085] In step S202, a compressed weight matrix in the compressed target model is identified;

[0086] In step S203, the compressed weight matrix is ​​decompressed.

[0087] In some embodiments, decompressing a compressed weight matrix includes: obtaining a first identifier and a second identifier of the compressed weight matrix; determining a vector set of the compressed weight matrix based on the first identifier; identifying the sequence number and vector ratio of the reorganized vector in the second identifier; reading the reorganized vector from the vector set based on the sequence number of the reorganized vector; and restoring the weight matrix of the target model based on the reorganized vector.

[0088] The first and second identifiers are key information in the decompression process. The first identifier is the sequence number of the vector set, used to locate the vector set to which the compressed weight matrix belongs. The second identifier contains information such as the sequence number and vector ratio of the reorganized vector, used to recover the reorganized vector from the vector set. During storage, these identifiers are stored in association with the compressed weight matrix. During decompression, these identifiers are first read from the storage medium to ensure accurate retrieval of the key data required for decompression.

[0089] Specifically, after obtaining the vector set sequence number corresponding to each compressed weight matrix, the storage system's indexing mechanism is used to locate the corresponding vector set within the previously read I vector sets. Once the vector set is located, the specific reorganized vector is read from the vector set based on the sequence number of each reorganized vector. A vector set is typically an ordered group of vectors, and the sequence number acts as an index into the group. Using this index, the corresponding reorganized vector can be quickly found from the base vector set.

[0090] Furthermore, the embodiment of the present application can utilize the read reorganized vectors and vector ratios to restore the weight matrix of the target model, thereby ensuring the accuracy of the data.

[0091] In summary, the target model compressed by the model compression storage method involved in the embodiment of the present application effectively reduces the space occupied by the model on the storage device. In particular, the compressed weight matrix is ​​stored in a manner such as a vector set, a first identifier, and a second identifier, thereby avoiding the storage of a large amount of redundant data and enabling the model to be smoothly deployed on devices with limited storage resources, such as mobile terminals, edge computing devices, etc. In the decompression stage, the vector set, the sequence number of the reorganized vector, and the vector ratio can be quickly determined based on the first identifier and the second identifier, thereby efficiently reading the reorganized vector from the vector set and restoring the weight matrix. Compared with the traditional decompression method, this precise search and recovery mechanism based on identifiers greatly shortens the decompression time and improves the startup speed and responsiveness of the model.

[0092] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0093] The embodiment of the present application also provides a model compression storage device, such as Figure 8 As shown, the model compression storage device 10 includes: a decomposition module 101, a generation module 102 and a storage module 103.

[0094] Among them, the decomposition module 101 is used to decompose the weight matrix of the target model into a combination of multiple target vectors; the generation module 102 is used to reorganize the combination of multiple target vectors and generate a vector set based on the reorganized vectors; the storage module 103 is used to store the compressed weight matrix based on the reorganized vectors and the vector set.

[0095] In some embodiments, the decomposition module 101 is further used to obtain a target decomposition dimension of the target model; and decompose the weight matrix into a combination of multiple target vectors according to the target decomposition dimension.

[0096] In some embodiments, the model compression storage device also includes: a grouping module for obtaining the number of weight matrices before decomposing the weight matrix of the target model into a combination of multiple target vectors; if there are multiple weight matrices, the multiple weight matrices are grouped according to the target decomposition dimension.

[0097] In some embodiments, the grouping module is further configured to group weight matrices of the same dimension into one group on the target decomposition dimension.

[0098] In some embodiments, the target decomposition dimension includes at least one of a row dimension and a column dimension.

[0099] In some embodiments, the generating module is further configured to determine a sequence number of the reorganized vector in the vector set; and generate the vector set according to the reorganized vector and the sequence number of the reorganized vector.

[0100] In some embodiments, the storage module 103 is further configured to identify a first identifier of a vector set; represent multiple target vectors using the reorganized vectors; and store the vector set of the compressed weight matrix, the first identifier, and a second identifier representing multiple target vectors.

[0101] In some embodiments, the storage module 103 is further configured to calculate second identifiers of the plurality of target vectors based on the reorganized vectors.

[0102] In some embodiments, the storage module 103 is further used to calculate the Euclidean distance between the target vector and the recombined vector; determine the recombined vector that matches the target vector based on the Euclidean distance; identify the sequence number of the recombined vector that matches the target vector; calculate the vector ratio of the target vector and the recombined vector that matches the target vector; and generate second identifiers of multiple target vectors based on the sequence number and vector ratio of the recombined vector that matches the target vector.

[0103] In some embodiments, the storage module 103 is further configured to obtain a sequence number of the vector set; and use the sequence number of the vector set as a first identifier of the vector set.

[0104] It should be noted that, for the description of the features in the embodiment corresponding to the model compression storage device, reference can be made to the relevant description of the embodiment corresponding to the model compression storage method, which will not be repeated here.

[0105] According to the model compression storage device proposed in the embodiment of the present application, the weight matrix of the target model is split into a combination of multiple target vectors, the combination of multiple target vectors is reorganized, and then a vector set is generated based on the reorganized vectors. The reorganization process can optimize and integrate the target vectors, and the generated vector set can more efficiently represent the original weight matrix. The compressed weight matrix is ​​stored based on the reorganized vectors and the vector set. Thus, the structural information inside the weight matrix can be fully mined, avoiding the direct storage of the entire weight matrix, thereby reducing storage space. Therefore, the problem of model storage occupying a large amount of storage space in the related art can be solved.

[0106] This application also provides a model decompression device 20, such as Figure 9 As shown, it includes: an acquisition module 201, an identification module 202 and a decompression module 203.

[0107] Among them, the acquisition module 201 is used to obtain the compressed target model, wherein the compressed target model is compressed using the model compression storage device of the above embodiment; the identification module 202 is used to identify the compressed weight matrix in the compressed target model; the decompression module 203 is used to decompress the compressed weight matrix.

[0108] In some embodiments, the decompression module 203 is further used to obtain a first identifier and a second identifier of the compressed weight matrix; determine a vector set of the compressed weight matrix based on the first identifier; identify the sequence number and vector ratio of the reorganized vector in the second identifier; read the reorganized vector from the vector set based on the sequence number of the reorganized vector; and restore the weight matrix of the target model based on the reorganized vector.

[0109] It should be noted that the description of the features in the embodiment corresponding to the model decompression device can be found in the relevant description of the embodiment corresponding to the model decompression method, and will not be repeated here.

[0110] According to the model decompression device proposed in the embodiments of this application, during the decompression phase, the vector set, the sequence number of the reorganized vector, and the vector ratio can be quickly determined based on the first and second identifiers, thereby efficiently reading the reorganized vector from the vector set and restoring the weight matrix. Compared to traditional decompression methods, this precise identifier-based search and recovery mechanism significantly shortens decompression time and improves the model's startup speed and responsiveness.

[0111] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above-mentioned model compression storage method and model decompression method embodiments.

[0112] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps of any one of the above-mentioned model compression storage method and model decompression method embodiments when run.

[0113] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0114] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above-mentioned model compression and storage methods and model decompression method embodiments are implemented.

[0115] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0116] The above is a detailed introduction to a model compression storage method and a model decompression method provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A model compression storage method, characterized in that: The method is used in an electronic device, the electronic device includes a storage space, and the compression model is stored in the storage space, wherein the method includes: Decomposing a weight matrix of the target model into a combination of multiple target vectors according to a target decomposition dimension of the target model, wherein when the input of the target model is an image, the image is divided into image slices, each image slice corresponds to a word element embedding, and the target decomposition dimension includes at least one of a row dimension and a column dimension. After decomposition according to the row dimension, each row vector is stored separately; decomposition according to the column dimension is performed to store and process the input information; Recombining the combination of the plurality of target vectors and generating a vector set based on the recombined vectors; Compressing the weight matrix of the target model based on the reorganized vector and the vector set, and storing the compressed weight matrix in the storage space according to a target storage structure, wherein the target storage structure is determined based on the target decomposition dimension, the vectors in the vector set are shared, the basis vectors are shared by multiple target vectors, and the basis vectors, coefficients, and index information are stored without storing the complete weight matrix; The storing of the compressed weight matrix based on the reorganized vector and the vector set includes: identifying a first identifier of the vector set; calculating a second identifier of the plurality of target vectors based on the reorganized vector; and storing the vector set of the compressed weight matrix, the first identifier, and a second identifier representing the plurality of target vectors. Calculating the second identifiers of the multiple target vectors based on the reorganized vectors includes: calculating the Euclidean distance between the target vector and the reorganized vector; determining the reorganized vector that matches the target vector based on the Euclidean distance; identifying the sequence number of the reorganized vector that matches the target vector; calculating the vector ratio between the target vector and the reorganized vector that matches the target vector; and generating the second identifiers of the multiple target vectors based on the sequence number of the reorganized vector that matches the target vector and the vector ratio.

2. The model compression storage method according to claim 1, characterized in that: Before decomposing the weight matrix of the target model into a combination of multiple target vectors, it also includes: Obtaining the number of the weight matrix; If there are multiple weight matrices, the multiple weight matrices are grouped according to the target decomposition dimension.

3. The model compression storage method according to claim 2, characterized in that: The grouping of the multiple weight matrices according to the target decomposition dimension includes: On the target decomposition dimension, weight matrices with the same dimension are divided into a group.

4. The model compression storage method according to claim 2 or 3, characterized in that: The target decomposition dimension includes at least one of a row dimension and a column dimension.

5. The model compression storage method according to claim 1, characterized in that: Generating a vector set according to the reorganized vector includes: Determining the sequence number of the reorganized vector in the vector set; A vector set is generated according to the reorganized vector and the sequence number of the reorganized vector.

6. The model compression storage method according to claim 1, characterized in that: The first identifier for identifying the vector set includes: Obtaining the sequence number of the vector set; The sequence number of the vector set is used as the first identifier of the vector set.

7. A model decompression method, characterized in that: The method is used in an electronic device, the electronic device includes a storage space, and the model is decompressed from the storage space, wherein the method includes: Acquire a compressed target model from the storage space, wherein the compressed target model is compressed using the model compression storage method according to any one of claims 1 to 6; identifying a compressed weight matrix in the compressed target model; A first identifier and a second identifier of the compressed weight matrix are obtained, index information is constructed based on the first identifier and the second identifier, and the compressed weight matrix is ​​decompressed based on the index information.

8. The model decompression method according to claim 7, characterized in that: The decompressing the compressed weight matrix includes: Determining a vector set of the compressed weight matrix according to the first identifier; Identifying the sequence number and vector ratio of the recombined vector in the second identifier; Reading a reorganized vector from the vector set based on the sequence number of the reorganized vector; The weight matrix of the target model is restored according to the reorganized vector.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the model compression and storage method according to any one of claims 1 to 6 and the model decompression method according to claim 7 or 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the model compression storage method according to any one of claims 1 to 6 and the model decompression method according to claim 7 or 8 are implemented.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the model compression and storage method according to any one of claims 1 to 6 and the model decompression method according to claim 7 or 8 are implemented.

Images