Model compression storage method, model decompression method, electronic equipment and medium

Abstract

The invention discloses a model compression storage method, a model decompression method, electronic equipment and a medium, and relates to the technical field of data storage, and the method comprises the steps: splitting a weight matrix of a target model into a combination of a plurality of target vectors, carrying out the recombination operation of the combination of the plurality of target vectors, and then generating a vector set according to the recombined vectors. The target vector can be optimized and integrated in the recombination process, the generated vector set can more efficiently represent the original weight matrix, and the compressed weight matrix is stored based on the recombined vector and the vector set, so that the internal structure information of the weight matrix can be fully mined, the direct storage of the whole weight matrix is avoided, and the reliability of the whole weight matrix is improved. Therefore, the storage space is reduced. Therefore, the problem that model storage occupies a large amount of storage space in related technologies can be solved.

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and in particular to a model compression storage method, a model decompression method, an electronic device and a medium. Background Art

[0002] As artificial intelligence technology develops rapidly, the Transformer architecture model has been widely used in many fields such as natural language processing and computer vision due to its excellent performance and has broad prospects. However, as the scale of the model continues to expand and the application scenarios become more complex, its storage requirements have become a key issue hindering its further development and application. Traditional model storage methods usually take up a lot of storage space, which places strict requirements on the capacity and performance of storage devices.

[0003] Related technologies propose to perform low-rank decomposition of the model's linear layer parameters and jointly quantize the activations and weights to reduce the number of parameters with low-bit representation; the intermediate activations after low-rank factor calculation are also quantized to low bits to achieve hardware-friendly model low-rank quantization acceleration, improve prediction and reasoning speed, and reduce storage requirements.

[0004] However, the focus of related technologies is on low-rank quantization, model acceleration, and accuracy recovery. Reducing storage requirements mainly relies on quantization and low-rank decomposition. However, in the Transformer architecture model, there are a large number of parameter matrices, which occupy a large amount of storage space. Summary of the invention

[0005] The present application provides a model compression storage method, a model decompression method, an electronic device and a medium, so as to at least solve the problem in the related art that model storage occupies a large amount of storage space.

[0006] The present application provides a model compression storage method, comprising: decomposing a weight matrix of a target model into a combination of multiple target vectors; recombining the combination of multiple target vectors and generating a vector set based on the recombined vectors; and storing a compressed weight matrix based on the recombined vectors and the vector set.

[0007] The present application provides a model decompression method, comprising: obtaining a compressed target model, wherein the compressed target model is compressed using the model compression storage method of the above-mentioned embodiment; identifying a compressed weight matrix in the compressed target model; and decompressing the compressed weight matrix.

[0008] The present application also provides a model compression storage device, including: a decomposition module, used to decompose the weight matrix of the target model into a combination of multiple target vectors; a generation module, used to reorganize the combination of multiple target vectors and generate a vector set based on the reorganized vectors; a storage module, used to store the compressed weight matrix based on the reorganized vectors and the vector set.

[0009] The present application also provides a model decompression device, including: an acquisition module, used to acquire a compressed target model, wherein the compressed target model is compressed using the model compression storage device of the above-mentioned embodiment; an identification module, used to identify the compressed weight matrix in the compressed target model; and a decompression module, used to decompress the compressed weight matrix.

[0010] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing any one of the above-mentioned model compression storage methods and model decompression method steps when executing the computer program.

[0011] The present application also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, any one of the above-mentioned model compression storage methods and model decompression method steps is implemented.

[0012] The present application also provides a computer program product, including a computer program, which implements any of the above-mentioned model compression storage methods and model decompression method steps when the computer program is executed by a processor.

[0013] Through the present application, the weight matrix of the target model is split into a combination of multiple target vectors, the combination of multiple target vectors is reorganized, and then a vector set is generated based on the reorganized vector. The reorganization process can optimize and integrate the target vectors, and the generated vector set can more efficiently represent the original weight matrix. The compressed weight matrix is ​​stored based on the reorganized vector and the vector set, thereby fully mining the structural information inside the weight matrix, avoiding the direct storage of the entire weight matrix, thereby reducing the storage space. Therefore, the problem of model storage occupying a large amount of storage space in the related art can be solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 An example diagram of query embedding generation module execution provided by one embodiment of the present application; Figure 2 An example diagram of a key embedding generation module execution provided by one embodiment of the present application; Figure 3 An example diagram of executing a value embedding generation module provided in one embodiment of the present application; Figure 4 An example diagram of sharing the same input embedding execution provided for an embodiment of the present application; Figure 5 An example diagram of a feedforward neural network layer execution provided for one embodiment of the present application; Figure 6 A flowchart of a model compression storage method provided in an embodiment of the present application; Figure 7 A flowchart of a model decompression method provided in an embodiment of the present application; Figure 8 A block diagram of a model compression storage device provided in an embodiment of the present application; Fig. 9 A block diagram of a model decompression device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0017] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0018] In the process of continuous development in the field of deep learning, the scale and complexity of models are increasing. Traditional model storage methods face severe challenges. Since they require a large amount of storage space, this not only puts forward high requirements on the capacity and performance of storage devices, but also limits the application and deployment of models in resource-constrained environments to a certain extent. In order to meet this challenge, low-bit quantization methods have emerged. The core of low-bit quantization methods lies in processing the values ​​in the model. In deep learning models, parameters are usually stored in the form of 32-bit floating point numbers. Although this can provide higher accuracy, it also occupies more storage space. Low-bit quantization technology quantizes these values ​​from 32-bit floating points to 8 bits or even lower bits. From a technical principle point of view, low-bit quantization is achieved by discretizing the original values. Taking 8-bit quantization as an example, it divides the original continuous value range into 256 discrete levels, and each level is represented by an 8-bit binary number. In this way, when storing, each parameter only takes up 8 bits of storage space, which significantly reduces the storage space compared to 32-bit floating point numbers.

[0019] On the other hand, from a structural perspective, weight matrices often show specific rules and patterns. In the weight matrix of the convolutional layer of a convolutional neural network, there may be similar structures between different convolution kernels, or there may be obvious linear correlations between certain rows and columns. These structural features provide an opportunity for the application of low-order matrix expression technology. Therefore, low-order matrix expression has also become an important means to reduce storage requirements. Using low-order matrix expression technology, the originally high-dimensional and complex weight matrix can be decomposed into a combination of low-order matrices. Therefore, after completing the low-order matrix decomposition of the weight matrix, only the relevant information of these low-order matrices needs to be saved in the storage link, which greatly reduces the storage overhead compared to directly storing the original weight matrix.

[0020] The embodiment of the present application can start from a new perspective. In the model based on the Transformer architecture, its internal structure is complex and has many parameters. Among them, a large number of parameter matrices have become a storage burden. A large number of multi-head attention mechanisms and feedforward neural network layers involve multiple parameter matrices in each layer. The scale of these parameter matrices is often tens of thousands of rows and columns. Under the traditional storage method, it takes a lot of storage space to store these matrices completely. Therefore, the embodiment of the present application disassembles these parameter matrices by row or column, so that they are represented as a combination of several vectors (row vectors or column vectors). This deep analysis and reorganization of the matrix structure provides a new basis for subsequent storage optimization. Secondly, a vector set is generated according to the reorganized vector. The reorganization process can optimize and integrate the target vector, and the generated vector set can more efficiently represent the original weight matrix. The compressed weight matrix is ​​stored based on the reorganized vector and the vector set, avoiding the direct storage of the complete matrix, significantly reducing the storage requirements, especially when processing large-scale Transformer model weights, which can greatly save storage resources.

[0021] Furthermore, if these vectors can be expressed as a linear combination of basis vectors, that is, a vector can be expressed as a combination of a set of basis vectors and a set of coefficients, then each vector can be expressed using only one set of coefficients. Since the basis vectors can be shared by all vectors, they can be stored separately, thereby saving storage space.

[0022] In order to enable those skilled in the art to better understand the present application, the Transformer architecture model involved in the present application is explained, including (1) query embedding generation module; (2) key embedding generation module; (3) value embedding generation module; (4) feedforward neural network layer (also called feedforward neural network module). The details are as follows: (1) Description of the query embedding generation module like Figure 1 As shown, the query embedding generation module embeds its module input ( ) is converted to a query embedding ( ). Assume that the module input embedding yes A column matrix with 1 row and 1 column, that is ; Query Embedding yes A column matrix with 1 row and 1 column, that is ; The query embedding generation module contains the weight matrix yes OK, The matrix of columns, that is Then, the process of module input embedding generating module output query embedding through query embedding can be formulated as: .

[0023] (2) Description of the key embedding generation module like Figure 2 As shown, the key embedding generation module embeds its module input ( ) is converted to a key embedding ( ). Assume that the module input embedding yes A column matrix with 1 row and 1 column, that is ;Key embedding yes A column matrix with 1 row and 1 column, that is ; The key embedding generation module contains the weight matrix yes OK, The matrix of columns, that is Then, the process of module input embedding generating module output key embedding through key embedding can be formulated as: .

[0024] (3) Description of the value embedding generation module like Figure 3 As shown, the value embedding generation module embeds its module input ( ) is converted to a value embedding ( ). Assume that the input embedding yes A column matrix with 1 row and 1 column, that is ; Value embedding yes A column matrix with 1 row and 1 column, that is ; The weight matrix included in the value embedding generation module yes OK, The matrix of columns, that is Then, the process of module input embedding generating module output value embedding through value embedding can be formulated as: .

[0025] Furthermore, if Figure 4 As shown in Figure 2, within the Transformer-based model, the query embedding generation module, key embedding generation module, and value embedding generation module are always used as a whole, and they share the same input embedding ( ). When single-head attention is used, the input embedding ( ) are input into the query embedding generation module, key embedding generation module and value embedding generation module respectively; in addition, when multi-head attention is adopted, it includes multiple sets of query embedding generation modules, key embedding generation modules and value embedding generation modules, and the module input embedding ( ) are input into each set of query embedding generation module, key embedding generation module and value embedding generation module respectively.

[0026] It should be noted that the input embedding (X) of the above module can be the word embedding generated by the word embedding encoder when the word is input into the model, or it can be the embedding inside the model. Taking text input as an example, when the text is input into the model, the text is divided into several words, and then each word corresponds to a word embedding. When representing the above word embedding, then usually ;when When representing the internal embedding of the model, It should be noted that and The numerical relationship of does not affect the application scope of the present invention. When the input is an image, the image is divided into a number of image slices, and then the image slices correspond to a word element embedding.

[0027] (4) Feedforward neural network layer (weights are ) like Figure 5 As shown, the feedforward neural network layer embeds its input into ( ) is converted to an output embedding ( ). Assume that the input embedding yes A column matrix with 1 row and 1 column, that is ; Output embedding yes A column matrix with 1 row and 1 column, that is ; The weight matrix contained in the feedforward neural network layer yes OK, The matrix of columns, that is .So, In the embodiment of the present application, the feedforward neural network layer refers to the feedforward neural network layer inside each Transformer module, and its input embedding and output embedding The dimensions are the same, that is .

[0028] The present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0029] Specifically, Figure 6A flowchart of a model compression storage method provided in an embodiment of the present application.

[0030] like Figure 6 As shown, the model compression storage method includes the following steps: In step S101, the weight matrix of the target model is decomposed into a combination of multiple target vectors.

[0031] The target model may include the query embedding generation module, the key embedding generation module, the value embedding generation module and the feedforward neural network layer, each module including multiple weight matrices.

[0032] In some embodiments, decomposing a weight matrix of a target model into a combination of multiple target vectors includes: obtaining a target decomposition dimension of the target model; and decomposing the weight matrix into a combination of multiple target vectors according to the target decomposition dimension, wherein the target decomposition dimension includes at least one of a row dimension and a column dimension.

[0033] It is understandable that the embodiments of the present application can decompose these weight matrices according to the row dimension or column dimension so that they are represented as a combination of several vectors (row vectors or column vectors). Further, that is, a vector can be represented as a combination of a set of vector serial numbers and a set of coefficients, then each vector can be expressed by only one set of coefficients, thereby saving storage space. It should be noted that in the embodiments of the present application, the serial number represents the serial number of the vector in the vector set. The vector can be retrieved from the vector set through this serial number. Each row of data can be approximately restored based on the retrieved vectors and coefficients.

[0034] Specifically, the target decomposition dimension determines how the matrix will be split, including row dimension and column dimension. The row dimension can be the horizontal direction of the matrix, and each row represents a set of related data features; the column dimension can be the vertical direction of the matrix, and each column also contains specific data information. When the target decomposition dimension is the row dimension, the weight matrix will be split into multiple row vectors. Each row vector contains the element values ​​of the row on each column. If the target decomposition dimension is the column dimension, the weight matrix will be decomposed into multiple column vectors. Each column vector contains the element values ​​of the column on each row. Different decomposition methods will lead to different storage structures. For example, in the weight matrix of a neural network, rows may correspond to different neurons, and columns may correspond to different input features. After decomposition by row, each row vector can be stored separately, which is convenient for managing the information of each neuron; decomposition by column is more suitable for storing and processing the information of input features. Choosing a suitable decomposition method can reduce storage redundancy and improve storage efficiency.

[0035] Furthermore, before decomposing the weight matrix of the target model into a combination of multiple target vectors, it also includes: obtaining the number of weight matrices; if there are multiple weight matrices, grouping the multiple weight matrices according to the target decomposition dimension.

[0036] It is understandable that in the Transformer architecture model, weight matrices are widely distributed in various functional modules, such as query embedding generation module, key embedding generation module, value embedding generation module, and feedforward neural network layer. There are many weight matrices in different modules, and each has specific functions and properties. Accurately obtain their number and group multiple weight matrices to facilitate unified processing of matrices in the same group. They can be quickly read and calculated by group, enhance the model's operating performance, and provide strong support for the application of the model in complex scenarios.

[0037] In the actual execution process, if there are multiple weight matrices, the embodiment of the present application can divide the weight matrices with the same dimension into a group on the target decomposition dimension, so that they can be operated in a unified manner in subsequent processing to improve computing efficiency.

[0038] For example, in a complex deep learning model, there may be multiple weight matrices, and different weight matrices correspond to different layers or different functional modules of the model. Obtaining the number of weight matrices can give us a preliminary understanding of the scale and complexity of the model. For example, in a multi-layer convolutional neural network, the convolution kernel of each layer corresponds to a weight matrix. By counting the number of weight matrices, the embodiment of the present application can know how many convolutional layers the network has, thereby providing basic information for subsequent processing. In actual operation, the embodiment of the present application can obtain the number of weight matrices through the structural information of the model.

[0039] Specifically, the embedding generation module, key embedding generation module, value embedding generation module and weight matrix in the feedforward neural network layer of the target model are grouped. The grouping basis of the embodiment of the present application can be the row dimension of the weight matrix, that is, matrices with the same row dimension are grouped into the same group.

[0040] Assume that all the above weight matrices can be divided into groups, where the i-th group contains weight matrices, which have the same number of columns (i.e., the same row dimensions) and are , then the i-th group can be expressed as .

[0041] <1> in, represents the set of weight matrices of the i-th group, , , Respectively represent the number of rows of each weight matrix in the i-th group, represents the number of columns of all weight matrices in the i-th group.

[0042] Because each group of weight matrices uses the same compression method, in order to simplify the discussion, the embodiment of the present application can omit the superscript , a set of weight matrices is represented by the following formula.

[0043] <2> In step S102, a combination of multiple target vectors is reorganized, and a vector set is generated according to the reorganized vectors.

[0044] It is understandable that after decomposing the weight matrix into multiple target vectors, although these target vectors each contain specific information, they are relatively scattered, which is not conducive to overall management and further processing. The purpose of recombining multiple target vectors is to explore the internal connections and regularities between the target vectors, integrate the scattered information, and form a more representative and regular new vector, that is, the recombined vector. Doing so helps reduce data redundancy, improve data expression efficiency, and lay the foundation for subsequent generation of vector sets and further optimization of model storage and computing efficiency.

[0045] Specifically, the embodiment of the present application can be a set of weight matrices Decompose into (in, )indivual The set of n-dimensional row vectors , as shown in Formula 3 below: <3> in, represent The set of dimensional real row vectors, denote row vectors, represents the sequence number of the row vector, is a row vector in the set The index in .

[0046] In some embodiments, the combination of multiple target vectors is reorganized, including: setting a number threshold of central vectors according to the number of multiple target vectors; clustering the multiple target vectors according to the number threshold of central vectors; and using the central vector obtained by clustering as a reorganized vector.

[0047] In the embodiment of the present application, a large number of target vectors can be compressed into a small number of center vectors through clustering. This greatly reduces the amount of data to be stored and improves the storage efficiency of the data. When processing large-scale data sets, this data compression effect is particularly obvious and can significantly reduce storage costs. Clustering is the process of dividing similar target vectors into the same group. The number threshold of the center vector determines the number of clustering groups, which is as follows: The embodiment of the present application may use a clustering algorithm (such as a K-means clustering algorithm) to indivual A collection of dimensional vectors Processing is performed and the vectors are clustered into center vector, where In the embodiments of the present application , In the present embodiment, the clustering result is A vector is also called basis vectors, i.e., the reorganized vectors. Therefore, by setting the threshold of the number of center vectors according to the number of target vectors, the clustering effect can be flexibly controlled to avoid the clustering results being too dispersed or concentrated, so that the reorganized vectors can more accurately represent the characteristics of the original target vectors. Secondly, clustering multiple target vectors and using the center vectors obtained by clustering as the reorganized vectors can greatly reduce data redundancy and represent similar vectors with a small number of center vectors, thereby reducing storage requirements and improving storage efficiency.

[0048] In some embodiments, generating a vector set according to the reorganized vector includes: determining a sequence number of the reorganized vector in the vector set; and generating the vector set according to the reorganized vector and the sequence number of the reorganized vector.

[0049] It is understandable that after decomposing the weight matrix into multiple target vectors and reorganizing them, a series of reorganized vectors will be obtained. In order to effectively manage and use these reorganized vectors, it is necessary to determine a sequence number for each reorganized vector in the vector set. The sequence number can be assigned in the order in which the vectors are generated, or can be numbered in sequence after being sorted according to certain characteristic attributes of the vectors, without specific limitation.

[0050] Taking an array as an example, the process of generating a vector set is to store the reorganized vectors in the corresponding positions of the array in sequence according to their serial numbers. Assuming there are three reorganized vectors b1, b2, and b3, and their serial numbers are 1, 2, and 3, then in the array, b1 will be stored in the first position of the array, b2 in the second position, and b3 in the third position. When storing vectors, other relevant information of the vectors can also be stored at the same time, such as the dimension and data type of the vector, so that it can be processed more conveniently in subsequent use.

[0051] The set of reorganized vectors is called a vector set (also called a basis vector set), expressed as . Each set of weight matrices corresponds to a set of vectors, then The group weight matrix corresponds to These vector sets will be stored in the storage unit as a "vector lookup table". During the model inference process, the basis vectors are read from the vector lookup table and the target vectors in the weight matrix are restored using the vectors.

[0052] In step S103, the compressed weight matrix is ​​stored based on the reorganized vector and the vector set.

[0053] It is understandable that the weight matrix of the model is huge, and direct storage will take up a lot of space. Decomposing and reorganizing it into a vector set for storage can effectively reduce storage requirements. After the weight matrix is ​​decomposed into vectors by row or column, the reorganized vectors are obtained through a clustering algorithm to form a vector set. The vectors in these vector sets are shareable. For example, the basis vector can be shared by multiple target vectors. Only the basis vector and a small amount of coefficients and index information need to be stored, and there is no need to store the complete weight matrix, which significantly reduces the storage cost and makes it easy to deploy the model on devices with limited storage resources.

[0054] Therefore, the embodiment of the present application can express each column or row of the weight matrix as a combination of several vectors, which can fully explore the structural information inside the weight matrix, avoid direct storage of the entire weight matrix, thereby reducing storage space, and can effectively improve storage efficiency. It provides strong support for the application and promotion of the Transformer architecture model in various resource-constrained environments, and has important practical value.

[0055] In some embodiments, storing a compressed weight matrix based on a reorganized vector and a vector set includes: identifying a first identifier of the vector set; using the reorganized vector to represent multiple target vectors; storing the vector set of the compressed weight matrix, the first identifier, and a second identifier representing the multiple target vectors.

[0056] The embodiment of the present application can obtain the serial number of the vector set; using the serial number of the vector set as the first identification of the vector set can ensure the accuracy and consistency of the vector set during storage and use. Each vector set has a unique serial number, which can effectively avoid confusion or misreference caused by unclear identification.

[0057] The present application can disassemble the weight matrix by row or column through the above-mentioned embodiment, and perform a reorganization operation. The embodiment of the present application can extract information that can uniquely identify the target vector, i.e., the second identifier, from the reorganized vector. And adopt a method of storing a vector set, a sequence number of a vector set, and a second identifier representing multiple target vectors respectively. Through this separate storage, not only the internal structure information of the weight matrix is ​​captured more accurately, but also the storage method is greatly optimized, avoiding direct storage of the entire weight matrix, thereby effectively reducing storage space.

[0058] In some embodiments, multiple target vectors are represented according to the reorganized vector, including: calculating the second identifiers of the multiple target vectors according to the reorganized vector. In some embodiments, the second identifiers of the multiple target vectors are calculated according to the reorganized vector, including: calculating the Euclidean distance between the target vector and the reorganized vector; determining the reorganized vector that matches the target vector according to the Euclidean distance; identifying the serial number of the reorganized vector that matches the target vector; calculating the vector ratio of the target vector and the reorganized vector that matches the target vector; based on the serial number and vector ratio of the reorganized vector that matches the target vector, generating the second identifiers of the multiple target vectors. Thus, by calculating the Euclidean distance, determining the matching vector, identifying the serial number, and calculating the vector ratio to generate the second identifier, the target vector is accurately represented and efficiently indexed, which helps to reduce the storage amount, improve the efficiency of data retrieval and model calculation, and optimize the storage and operation performance of the Transformer architecture model.

[0059] The embodiment of the present application can be further explained by taking the target vector as a row vector as an example. If each row vector of the parameter matrix in the Transformer architecture model can be expressed as a combination of several basis vectors and corresponding coefficients, then when performing storage operations, each row vector can be represented by its corresponding coefficient and the serial number of the basis vector. The effect of reducing storage space can be achieved if and only if the storage space occupied by the coefficient and the serial number of the basis vector corresponding to each row vector is less than the space occupied by the original row vector. In the most ideal extreme case, each row vector only needs to use one coefficient and a corresponding serial number of the basis vector to complete the expression.

[0060] For a row vector set Any row vector in According to the Euclidean distance formula between vectors, we can get The basis vector with the closest Euclidean distance is calculated in , assuming that the reorganized vector is , then record its serial number .

[0061] Assume that the row vector , the nearest reorganized vector is ,So , represents the Euclidean distance. Therefore, .

[0062] After the above steps, each vector Can be expressed as a and a coefficient Because each set of weight matrices shares a set of basis vectors, there is no need to Direct Recording , only record the corresponding serial number Here you can see that each vector It can be expressed by two numbers and , significantly reducing the amount of storage.

[0063] According to the model compression storage method proposed in the embodiment of the present application, the weight matrix of the target model is split into a combination of multiple target vectors, the combination of multiple target vectors is reorganized, and then a vector set is generated based on the reorganized vector. The reorganization process can optimize and integrate the target vectors, and the generated vector set can more efficiently represent the original weight matrix. The compressed weight matrix is ​​stored based on the reorganized vector and the vector set, thereby fully mining the structural information inside the weight matrix, avoiding direct storage of the entire weight matrix, thereby reducing storage space. Therefore, the problem of model storage occupying a large amount of storage space in the related art can be solved.

[0064] Secondly, this application also provides a model decompression method, such as Figure 7 As shown, including: In step S201, a compressed target model is obtained, wherein the compressed target model is compressed using a model compression storage method.

[0065] In step S202, a compressed weight matrix in a compressed target model is identified; In step S203, the compressed weight matrix is ​​decompressed.

[0066] In some embodiments, decompressing a compressed weight matrix includes: obtaining a first identifier and a second identifier of the compressed weight matrix; determining a vector set of the compressed weight matrix based on the first identifier; identifying a sequence number and a vector ratio of a reorganized vector in the second identifier; reading a reorganized vector from the vector set based on the sequence number of the reorganized vector; and restoring a weight matrix of a target model based on the reorganized vector.

[0067] Among them, the first identifier and the second identifier are key information in the decompression process. The first identifier is the serial number of the vector set, which is used to locate the vector set to which the compressed weight matrix belongs. The second identifier contains information such as the serial number and vector ratio of the reorganized vector, which is used to restore the reorganized vector in the vector set. When storing, these identifiers are stored in association with the compressed weight matrix. When decompressing, these identification information is first read from the storage medium to ensure that the key data required for decompression can be accurately obtained.

[0068] Specifically, after obtaining the vector set serial number corresponding to each compressed weight matrix, the corresponding vector set is found in the I vector sets that have been read according to the index mechanism of the storage system. After locating the vector set, the specific reorganized vector is read from the vector set according to the serial number of each reorganized vector. Among them, the vector set is usually an ordered vector group, and the serial number is like the index of the vector group. Through this index, the corresponding reorganized vector can be quickly found from the base vector set.

[0069] Furthermore, the embodiment of the present application can utilize the read reorganized vectors and vector ratios to restore the weight matrix of the target model, thereby ensuring the accuracy of the data.

[0070] In summary, the target model compressed by the model compression storage method involved in the embodiment of the present application effectively reduces the space occupied by the model on the storage device. In particular, the compressed weight matrix is ​​stored in a manner such as a vector set, a first identifier, and a second identifier, thereby avoiding the storage of a large amount of redundant data, so that the model can be smoothly deployed on devices with limited storage resources, such as mobile terminals, edge computing devices, etc. And in the decompression stage, the vector set, the sequence number of the reorganized vector and the vector ratio can be quickly determined based on the first identifier and the second identifier, so as to efficiently read the reorganized vector from the vector set and restore the weight matrix. Compared with the traditional decompression method, this accurate search and recovery mechanism based on identifiers greatly shortens the decompression time and improves the startup speed and responsiveness of the model.

[0071] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0072] The embodiment of the present application also provides a model compression storage device, such as Figure 8 As shown, the model compression storage device 10 includes: a decomposition module 101, a generation module 102 and a storage module 103.

[0073] Among them, the decomposition module 101 is used to decompose the weight matrix of the target model into a combination of multiple target vectors; the generation module 102 is used to reorganize the combination of multiple target vectors and generate a vector set based on the reorganized vectors; the storage module 103 is used to store the compressed weight matrix based on the reorganized vectors and the vector set.

[0074] In some embodiments, the decomposition module 101 is further used to obtain a target decomposition dimension of the target model; and decompose the weight matrix into a combination of multiple target vectors according to the target decomposition dimension.

[0075] In some embodiments, the model compression storage device also includes: a grouping module, which is used to obtain the number of weight matrices before decomposing the weight matrix of the target model into a combination of multiple target vectors; if there are multiple weight matrices, the multiple weight matrices are grouped according to the target decomposition dimension.

[0076] In some embodiments, the grouping module is further used to group weight matrices with the same dimension into one group on the target decomposition dimension.

[0077] In some embodiments, the target decomposition dimension includes at least one of a row dimension and a column dimension.

[0078] In some embodiments, the generating module is further used to determine the sequence number of the reorganized vector in the vector set; and generate the vector set according to the reorganized vector and the sequence number of the reorganized vector.

[0079] In some embodiments, the storage module 103 is further used to identify a first identifier of a vector set; use the reorganized vector to represent multiple target vectors; and store the vector set of the compressed weight matrix, the first identifier, and a second identifier representing multiple target vectors.

[0080] In some embodiments, the storage module 103 is further configured to calculate second identifiers of the plurality of target vectors according to the reorganized vectors.

[0081] In some embodiments, the storage module 103 is further used to calculate the Euclidean distance between the target vector and the reorganized vector; determine the reorganized vector that matches the target vector based on the Euclidean distance; identify the sequence number of the reorganized vector that matches the target vector; calculate the vector ratio of the target vector and the reorganized vector that matches the target vector; and generate second identifiers of multiple target vectors based on the sequence number and vector ratio of the reorganized vector that matches the target vector.

[0082] In some embodiments, the storage module 103 is further configured to obtain a sequence number of the vector set; and use the sequence number of the vector set as a first identifier of the vector set.

[0083] It should be noted that the description of the features in the embodiment corresponding to the model compression storage device can refer to the relevant description of the embodiment corresponding to the model compression storage method, which will not be repeated here.

[0084] According to the model compression storage device proposed in the embodiment of the present application, the weight matrix of the target model is split into a combination of multiple target vectors, the combination of multiple target vectors is reorganized, and then a vector set is generated based on the reorganized vector. The reorganization process can optimize and integrate the target vectors, and the generated vector set can more efficiently represent the original weight matrix. The compressed weight matrix is ​​stored based on the reorganized vector and the vector set, thereby fully mining the structural information inside the weight matrix, avoiding direct storage of the entire weight matrix, thereby reducing storage space. Therefore, the problem of model storage occupying a large amount of storage space in the related art can be solved.

[0085] The present application also provides a model decompression device 20, such as Fig. 9 As shown, it includes: an acquisition module 201, an identification module 202 and a decompression module 203.

[0086] Among them, the acquisition module 201 is used to obtain the compressed target model, wherein the compressed target model is compressed using the model compression storage device of the above embodiment; the identification module 202 is used to identify the compressed weight matrix in the compressed target model; the decompression module 203 is used to decompress the compressed weight matrix.

[0087] In some embodiments, the decompression module 203 is further used to obtain a first identifier and a second identifier of the compressed weight matrix; determine a vector set of the compressed weight matrix based on the first identifier; identify the sequence number and vector ratio of the reorganized vector in the second identifier; read the reorganized vector from the vector set based on the sequence number of the reorganized vector; and restore the weight matrix of the target model based on the reorganized vector.

[0088] It should be noted that, for the description of the features in the embodiment corresponding to the model decompression device, reference can be made to the relevant description of the embodiment corresponding to the model decompression method, which will not be repeated here.

[0089] According to the model decompression device proposed in the embodiment of the present application, in the decompression stage, the vector set, the sequence number of the reorganized vector and the vector ratio can be quickly determined according to the first identifier and the second identifier, so as to efficiently read the reorganized vector from the vector set and restore the weight matrix. Compared with the traditional decompression method, this accurate search and recovery mechanism based on identifiers greatly shortens the decompression time and improves the startup speed and responsiveness of the model.

[0090] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above-mentioned model compression storage method and model decompression method embodiments.

[0091] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above-mentioned model compression storage method and model decompression method embodiments when running.

[0092] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0093] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above-mentioned model compression storage method and model decompression method embodiments are implemented.

[0094] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0095] The above is a detailed introduction to a model compression storage method and a model decompression method provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A model compression storage method, characterized in that: include: Decompose the weight matrix of the target model into a combination of multiple target vectors; Recombining the combination of the plurality of target vectors, and generating a vector set according to the recombined vectors; A compressed weight matrix is ​​stored based on the reorganized vector and the set of vectors.

2. The model compression storage method according to claim 1, characterized in that: The step of decomposing the weight matrix of the target model into a combination of multiple target vectors includes: Obtaining a target decomposition dimension of the target model; The weight matrix is ​​decomposed into a combination of multiple target vectors according to the target decomposition dimension.

3. The model compression storage method according to claim 2, characterized in that: Before decomposing the weight matrix of the target model into a combination of multiple target vectors, it also includes: Obtaining the number of the weight matrix; If there are multiple weight matrices, the multiple weight matrices are grouped according to the target decomposition dimension.

4. The model compression storage method according to claim 3, characterized in that: The grouping of the multiple weight matrices according to the target decomposition dimension includes: On the target decomposition dimension, weight matrices with the same dimension are divided into a group.

5. The model compression storage method according to any one of claims 2 to 4, characterized in that: The target decomposition dimension includes at least one of a row dimension and a column dimension.

6. The model compression storage method according to claim 1, characterized in that: The step of generating a vector set according to the reorganized vectors includes: Determine the sequence number of the reorganized vector in the vector set; A vector set is generated according to the reorganized vector and the sequence number of the reorganized vector.

7. The model compression storage method according to claim 1, characterized in that: The storing of the compressed weight matrix based on the reorganized vector and the vector set includes: A first identifier identifying the vector set; Representing the plurality of target vectors using the reorganized vectors; A vector set of compressed weight matrices, the first identifier, and a second identifier representing the plurality of target vectors are stored.

8. The model compression storage method according to claim 7, characterized in that: The step of representing the plurality of target vectors according to the reorganized vectors comprises: The second identifiers of the plurality of target vectors are calculated according to the reorganized vectors.

9. The model compression storage method according to claim 8, characterized in that: The step of calculating the second identifiers of the plurality of target vectors according to the reorganized vectors includes: Calculating the Euclidean distance between the target vector and the reorganized vector; Determine a recombined vector that matches the target vector according to the Euclidean distance; Identify the sequence number of the recombined vector that matches the target vector; Calculating a vector ratio of the target vector and a recombined vector matched by the target vector; Based on the sequence number of the reorganized vector matched by the target vector and the vector ratio, a second identifier of the plurality of target vectors is generated.

10. The model compression storage method according to claim 7, characterized in that: The first identification of the vector set includes: Obtaining the sequence number of the vector set; The sequence number of the vector set is used as the first identifier of the vector set.

11. A model decompression method, characterized in that: include: Acquire a compressed target model, wherein the compressed target model is compressed using the model compression storage method according to any one of claims 1 to 10; identifying a compressed weight matrix in the compressed target model; The compressed weight matrix is ​​decompressed.

12. The model decompression method according to claim 11, characterized in that: The decompressing the compressed weight matrix comprises: Obtain a first identifier and a second identifier of the compressed weight matrix; Determining a vector set of the compressed weight matrix according to the first identifier; Identify the sequence number and vector ratio of the reorganized vector in the second identifier; Reading a reorganized vector from the vector set based on the sequence number of the reorganized vector; The weight matrix of the target model is restored according to the reorganized vector.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, used to implement the steps of the model compression storage method according to any one of claims 1 to 10 and the model decompression method according to 11 or 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the model compression storage method according to any one of claims 1 to 10 and the model decompression method according to claim 11 or 12.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the model compression storage method according to any one of claims 1 to 10 and the model decompression method according to claim 11 or 12 are implemented.

Images