Data compression storage method and device, storage medium and electronic equipment
By storing machine learning model parameters in a distributed manner, the problem of excessive storage space requirements after model parameter conversion is solved, and efficient storage compression is achieved.
Patent Information
- Application Number
- CN202510890230.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-27
- Publication Date
- 2025-10-17
AI Technical Summary
In an environment with limited storage space, after the machine learning model parameters are converted into vectors in a preset ciphertext format, the storage space requirements increase dramatically, resulting in excessive storage pressure.
After converting the model parameters in the machine learning model into vectors in a preset ciphertext format, the same parameters are combined according to the position distribution and the parameter values and position distribution are stored to reduce duplicate storage.
It effectively compresses the storage space requirements after the model parameters are converted into preset ciphertext format vectors, saving a lot of storage space.
Smart Images
Figure CN120805997A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present specification relates to the technical field of computers, and in particular, to a data compression storage method and device, a storage medium, and an electronic device. BACKGROUND
[0002] In the context of today's digital era, maintaining personal privacy and protecting data security have become a hot topic, which has given rise to a new data technology - privacy computing. The biggest highlight of this technology is that it can perform relevant data analysis and processing in a secret state, ensuring the privacy and security of sensitive data during use. The fully homomorphic encryption technology (FHE) is one of the classic methods in the field of privacy computing, which can complete specific types of calculations on data in the state of data encryption as ciphertext, ensuring the correctness of the operation result while protecting the privacy of the data. With its continuous application in cloud computing, secure data sharing, and multi-dimensional information collaboration platforms, homomorphic encryption is gradually building a bridge to connect the needs of efficient data utilization and strict data protection, leading the field of secure computing to a new stage of development.
[0003] In some application scenarios, the requester can input the ciphertext input data encrypted by the fully homomorphic encryption technology into a machine learning model pre-trained by the service provider, process the ciphertext input data through the machine learning model, obtain a ciphertext processing result, and return the ciphertext processing result to the requester. The requester decrypts the ciphertext processing result to obtain a plaintext processing result. In this way, the requester can use the machine learning model of the service provider to obtain services without revealing the plaintext input data.
[0004] Generally, the format of the ciphertext input data is usually a vector of a preset ciphertext format, which requires that when the machine learning model is used to process the ciphertext input data, the model parameters in the machine learning model are also converted into a vector of the preset ciphertext format, so that the ciphertext input data of the same format can be processed. However, after converting the model parameters in the machine learning model into a vector of the preset ciphertext format, the storage space required by these model parameters will increase dramatically, which is a very difficult problem for implementation environments with limited storage space capabilities (such as ASIC, Chiplet, etc.). SUMMARY
[0005] The embodiments of the present specification provide a data compression storage method, device, storage medium, and electronic device to partially solve the problems existing in the prior art.
[0006] The embodiments of the present specification adopt the following technical solutions:
[0007] The data compression storage method provided by the present specification comprises:
[0008] obtain each model parameter in the machine learning model;
[0009] For each model parameter, determine the position distribution of elements with different values in the vector corresponding to the model parameter after the model parameter is converted into the vector in the preset ciphertext format.
[0010] group the model parameters with the same position distribution as a same parameter group, and for each parameter group, store the parameter values of the model parameters in the parameter group and the position distribution of elements with different values in the vector corresponding to any model parameter in the parameter group.
[0011] The data compression storage device provided in the specification comprises:
[0012] an obtaining module configured to obtain each model parameter in the machine learning model;
[0013] a determining module configured to, for each model parameter, determine the position distribution of elements with different values in the vector corresponding to the model parameter after the model parameter is converted into the vector in the preset ciphertext format.
[0014] a compression storage module configured to group the model parameters with the same position distribution as a same parameter group, and for each parameter group, store the parameter values of the model parameters in the parameter group and the position distribution of elements with different values in the vector corresponding to any model parameter in the parameter group.
[0015] The computer readable storage medium provided in the specification stores a computer program, and the computer program is executed by a processor to implement the data compression storage method.
[0016] The electronic device provided in the specification comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the data compression storage method when executing the program.
[0017] The above at least one technical solution adopted by the embodiments of the specification can achieve the following beneficial effects:
[0018] The embodiment of the present specification discloses a data compression storage method, for each model parameter in a machine learning model, determining the position distribution of elements with different values in a vector after the model parameter is converted into a preset ciphertext format, regarding the model parameters with the same position distribution as a same parameter group, for each parameter group, storing the parameter values of the model parameters in the parameter group and the position distribution of elements with different values in the vector corresponding to any model parameter in the parameter group. Since the method can not store the vector in the preset ciphertext format corresponding to each model parameter, only one position distribution and single-point values of the model parameters in the same parameter group need to be stored, the storage space required after the model parameters in the machine learning model are converted into the vector in the preset ciphertext format can be effectively compressed. BRIEF DESCRIPTION OF DRAWINGS
[0019] The drawings described herein are used to provide further understanding of the present specification, constitute a part of the present specification, the illustrative embodiments of the present specification and the description thereof are used to explain the present specification, and do not constitute improper limitation on the present specification. In the drawings:
[0020] Figure 1 A flow chart of a data compression storage method provided by the embodiment of the present specification;
[0021] Figure 2 An illustrative diagram of processing input data which is also a vector in simd format using a vector in simd format converted from a model parameter provided by the embodiment of the present specification;
[0022] Figure 3 An illustrative diagram of a normal convolution process of an input image using a convolution kernel provided by the embodiment of the present specification;
[0023] Figure 4 An illustrative diagram of a convolution process after each element in an input image and a convolution kernel is converted into simd format provided by the embodiment of the present specification;
[0024] Figure 5 An illustrative diagram of determining the position of an element in a vector in a preset ciphertext format after the element is converted into the vector according to the position of the element in a convolution kernel provided by the embodiment of the present specification;
[0025] Figure 6 An illustrative diagram of a preset ciphertext format vector whose dimension is not equal to the dimension of a one-dimensional array corresponding to ciphertext input data provided by the embodiment of the present specification;
[0026] Figure 7 An illustrative diagram of a simd format vector corresponding to a target parameter to be restored provided by the embodiment of the present specification;
[0027] Figure 8 Another schematic diagram of reducing the target parameter corresponding to the simd format of the vector is provided for the embodiments of the present specification.
[0028] Figure 9 A schematic diagram of a data compression storage device is provided for the embodiments of the present specification.
[0029] Figure 10 A schematic diagram of an electronic device is provided for the embodiments of the present specification. DETAILED DESCRIPTION
[0030] In order to make the purpose, technical scheme and advantages of the present specification clearer, the technical scheme of the present specification will be described in detail below in combination with the specific embodiments of the present specification and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present specification, not all the embodiments. Based on the embodiments in the present specification, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present specification.
[0031] The technical scheme provided by each embodiment of the present specification will be described in detail below in combination with the drawings.
[0032] Figure 1 A flowchart of a data compression storage method is provided for the embodiments of the present specification, which specifically includes the following steps:
[0033] S100: Obtain each model parameter in the machine learning model.
[0034] In the embodiments of the present specification, the device for compressing and storing data by using the method as shown in Figure 1 The device for compressing and storing data by using the method can be an electronic device deploying a machine learning model for providing services, such as a cloud device deploying a machine learning model, etc. The following will be described only by taking a cloud server as an example.
[0035] The cloud server has a pre-trained machine learning model deployed in advance, which is used to provide inference services for users, that is, the cloud server receives input data sent by a user, inputs the input data sent by the user into the machine learning model, obtains a processing result output by the machine learning model, and returns the processing result to the user.
[0036] In the secure inference scenario, the input data sent by the user to the cloud server is not plaintext input data, but ciphertext input data encrypted by a fully homomorphic encryption algorithm. The ciphertext input data is usually a vector in a preset ciphertext format corresponding to the fully homomorphic encryption algorithm. Therefore, to process the ciphertext input data, the cloud server also needs to convert the model parameters in the machine learning model into a vector in the preset ciphertext format. To compress the storage space required after converting the model parameters into a vector in the preset ciphertext format, the cloud server can first determine all the model parameters in the machine learning model.
[0037] The preset ciphertext format corresponding to the CKKS homomorphic encryption algorithm can be a simd format.
[0038] For example, when the machine learning model deployed in the cloud server is a CNN model (such as a convolutional neural network model such as Resnet) for processing images, the data that the user needs to process is an image. The image is generally a matrix with a certain length and a certain width. After the image is encrypted by CKKS and converted into a simd format vector, the image is converted into a one-dimensional array. The dimension of the simd format vector can be the same as that of the one-dimensional array, or it can be different. For convenience of description, the dimension of the simd format vector is the same as that of the one-dimensional array is taken as an example for description. The cloud server also needs to convert the model parameters in the machine learning model into the same simd format vector to process the CKKS ciphertext input data (i.e., the simd format vector obtained by encrypting the image as input data by CKKS and converting it). Figure 2
[0039] Figure 2 The use of converting model parameters into simd format vectors to process input data in the same simd format provided by the embodiments of the present disclosure is illustrated in Figure 2 In the figure, vector A is a simd format vector corresponding to ciphertext input data, and the one-dimensional array corresponding to the ciphertext input data has 8 elements A0-A7. Vector B is a simd format vector obtained by converting model parameters, which is also an 8-dimensional vector B0-B7. Figure 2 In the figure, when vector B corresponding to the model parameters is used to process vector A corresponding to the ciphertext input data, if the model parameters need to be added to the ciphertext input data, vector A and vector B can be directly added to each other. If the model parameters need to be multiplied by the ciphertext input data, vector B and vector B can be directly multiplied to each other.
[0040] S102: For each model parameter, determine the position distribution of elements with different values in the vector after the model parameter is converted into a vector in a preset ciphertext format.
[0041] After all the model parameters in the machine learning are determined, the cloud server can determine, for each model parameter, the position distribution of elements with different values in the vector after the model parameter is converted into a vector in a preset ciphertext format.
[0042] It should be noted that the conversion of the model parameter into the vector in the preset ciphertext format is not the encryption of the model parameter by using the fully homomorphic encryption algorithm, but the direct conversion of the plaintext model parameter into the preset ciphertext format, that is, the change of the format of the plaintext model parameter, which is still the plaintext model parameter, not the ciphertext model parameter. In addition, in step S102, the cloud server only determines the position distribution of elements with different values in the vector, but does not convert the model parameter into the corresponding vector in the preset ciphertext format.
[0043] Specifically, when the machine learning model is a CNN model, each element in each convolution kernel in the CNN model is a model parameter in the machine learning model. Then, for a model parameter, the cloud server determines the position distribution of elements with different values in the vector after the model parameter is converted into a vector in a preset ciphertext format, including: for each element in the convolution kernel, determining the position of the element in the convolution kernel, and determining the position of the element in the vector after the element is converted into the vector in the preset ciphertext format according to the position of the element in the convolution kernel; and determining the position distribution of elements with different values in the vector according to the position of the element in the vector, as shown in Figure 3 .
[0044] Figure 3 A schematic diagram of the normal convolution process of the input image by using the convolution kernel is provided for the embodiments of the present specification.
[0045] In Figure 3 , the input image is a 4x4 three-channel matrix A, and the elements in the matrix A are A00-A33. The matrix A in different channels is represented by different shades. The convolution kernel is a 3x3 convolution kernel a-c, and the elements in the convolution kernel a are fa1-fa9, the elements in the convolution kernel b are fb1-fb9, and the elements in the convolution kernel c are fc1-fc9. For convenience of description, Figure 3 , the input image is a three-channel, and the convolution kernel is taken as an example for description. After the normal convolution process shown in Figure 3 , the output result is a matrix B, and the elements in the matrix B are B00-B33.
[0046] And when the Figure 3 The input image shown in is encrypted and converted into a vector in SIMD format, and each element in the convolution kernel is also converted into a vector in SIMD format, as shown in Figure 3 The convolution operation shown becomes Figure 4 Vector addition and multiplication operations shown.
[0047] Figure 4 A schematic diagram of the convolution process after converting the input image and each element in the convolution kernel into SIMD format is provided in the embodiments of this specification.
[0048] exist Figure 4 In the output vector B, Figure 3 The ordering of the elements in the matrix B shown forms an array of 16-dimensional vectors, and the vector A is Figure 3 The matrix A is encrypted by CKKS and converted into a vector in simd format. Figure 3 Each element in the three convolution kernels shown is also converted into a vector in SIMD format (i.e. Figure 4 The unshaded vector to the right of the equal sign). Thus, Figure 4 The output vector B in can be directly expressed as the vector A rotated right by 1 unit, multiplied by the vector in simd format corresponding to a certain element, and then added (for convenience, Figure 4 In the equation, vector A and the vector obtained by rotating vector A right by several units are both denoted as vector A).
[0049] visible, Figure 4 After the elements with different positions in the convolution kernel are converted into vectors in SIMD format, the positions of elements with different values in the vector are distributed differently. After the elements with the same position in the convolution kernel are converted into vectors in SIMD format, the positions of elements with different values in the vector are distributed the same. For example, Figure 5 shown.
[0050] Figure 5 A schematic diagram of determining the position of an element in a convolution kernel after converting the element into a vector in a preset ciphertext format according to the position of the element in the convolution kernel provided in an embodiment of this specification, Figure 4 The convolution kernels a to c in the convolution kernel have the same size, and the elements fa1 in the convolution kernel a, fb1 in the convolution kernel b, and fc1 in the convolution kernel c have the same position in their respective convolution kernels. Therefore, in Figure 5 In the smid format vectors corresponding to the three convolution kernels shown, the positions of elements with different values in the vector are exactly the same.
[0051] Because in Figure 5In the shown vector, the value of one vector has only two possibilities, one is 0, and the other is the element in the convolution kernel. Therefore, the cloud server can determine the position distribution of the elements with value 0 and the elements with value not 0 in the vector after converting each model parameter into the vector in the preset ciphertext format.
[0052] S104: Grouping each model parameter with the same position distribution as the same parameter group, and storing the parameter values of each model parameter in the parameter group and the position distribution of the elements with different values in the vector corresponding to any model parameter in the parameter group.
[0053] After the cloud server determines the position distribution corresponding to each model parameter in the machine learning model, it can group each model parameter with the same position distribution into the same parameter group and store the parameter values of each model parameter in the parameter group. Since the position distributions corresponding to all model parameters in the same parameter group are the same, in addition to storing the parameter values of each model parameter in the parameter group, the cloud server in the embodiments of the present specification only needs to store the position distribution corresponding to one model parameter in the parameter group.
[0054] In storing the position distribution corresponding to one model parameter in the parameter group, the cloud server can store a vector composed of 0 and 1, which is also in the preset ciphertext format, to represent the position distribution corresponding to all model parameters in the parameter group. Hereinafter, the preset ciphertext format vector composed of 0 and 1 is referred to as a position distribution vector. Specifically, the cloud server can determine the position of each element with a non-0 value in the vector, determine the position distribution vector corresponding to the element with a non-0 value according to the position of the element with a non-0 value in the vector, and store it, wherein the position distribution vector has the same dimension as the preset ciphertext format vector, the element at the position of the element with a non-0 value in the position distribution vector has a value of 1, and the elements at other positions have a value of 0, as shown in Figure 5 .
[0055] In Figure 5 , since the positions of the elements fa1, fb1 and fc1 in the respective simd format vectors are exactly the same, the cloud server groups these three elements into the same parameter group and stores the single-point parameter values of the elements fa1, fb1 and fc1. Then, according to the positions of any one of the three elements in the corresponding simd format vector, i.e., the positions of the 6th-8th elements, the 10th-12th elements and the 14th-16th elements in the simd format vector, the cloud server sets the elements at these positions to 1 and the elements at other positions to 0 to determine the position distribution vector as shown in Figure 5 , and stores it.
[0056] It can be seen that after converting a model parameter into a preset ciphertext format vector, there are a large number of repeated value elements in the vector, and for each model parameter converted into a preset ciphertext format vector with the same position distribution, if the vectors corresponding to the model parameters are directly stored, a large amount of storage space will be occupied. For example, in Resnet50, if all elements in all convolution kernels are directly converted into simd format vectors and stored, the storage space occupied by the simd format vectors corresponding to the model parameters will exceed 500 GB, even reaching the order of 1 TB. Through the above method, the cloud server does not need to store the preset ciphertext format vectors corresponding to each model parameter in the machine learning model, but for a parameter group composed of model parameters with the same position distribution, only a position distribution vector representing the position distribution of each model parameter in the preset ciphertext format vector in the parameter group is stored, and the single-point parameter values of all model parameters are stored, which can greatly compress the storage space required after the model parameters are converted into vectors.
[0057] Still taking Figure 5 as an example, if the three elements fa1, fb1 and fc1 are directly converted into simd format vectors and stored, assuming that the storage space occupied is O, after compression storage by the method provided in the present specification, the storage space occupied by the single-point parameter values of the three elements can be almost negligible, and only one simd format position distribution vector needs to be stored additionally, and the required storage space is only O / 3. That is, in a machine learning model, the more the number of model parameters with the same position distribution determined by step S102, the more storage space can be saved by the method provided in the present specification. In actual application scenarios, the number of model parameters with the same position distribution in a Resnet is often as high as hundreds or thousands, and through the above method provided in the present specification, the required storage space is only one hundredth or even one thousandth of the storage space of directly storing simd vectors.
[0058] Further, in the embodiments of the present disclosure, the dimension of the vector of the preset ciphertext format depends on the polynomial length N in the used homomorphic encryption algorithm. Specifically, when the homomorphic encryption algorithm is the CKKS algorithm, if the polynomial length is N, then the dimension of the vector of the simd format (and the preset ciphertext format) is N / 2. If the dimension of the vector of the preset ciphertext format is not the same as the dimension of the one-dimensional array corresponding to the ciphertext input data, in step S104, when determining the model parameters with the same position distribution, for any two model parameters, if the positions of the elements with different values in the two vectors are the same after the two model parameters are converted into the vectors of the preset ciphertext format, or the positions of the elements with different values in one of the vectors are the same as the positions of the elements with different values in the other vector after the one of the vectors is rotated by n units in a specified direction, then the two model parameters are determined as the model parameters with the same position distribution, and the two model parameters are divided into the same parameter group. Here, n is an integer multiple of the dimension of the one-dimensional array corresponding to the ciphertext input data, and the specified direction can be right or left. In addition, when storing the position distribution corresponding to the parameter group, only the position distribution vector corresponding to one of the model parameters in the parameter group can still be stored, and for the other model parameters that need to be rotated by n units in the specified direction to have the same position distribution as the position distribution vector, the number of units needed to be rotated in the specified direction relative to the position distribution vector can be stored, such as Figure 6 .
[0059] Figure 6 The schematic diagram of the dimension of the vector of the preset ciphertext format provided by the embodiments of the present disclosure being different from the dimension of the one-dimensional array corresponding to the ciphertext input data is shown in Figure 6 . The one-dimensional array corresponding to the ciphertext input data is the same as Figures 3-5 , and is 16-dimensional, but the polynomial length N in the CKKS algorithm is 64, that is, the dimension of the vector of the simd format is 32-dimensional. The positions of the elements with different values in the vector of the simd format corresponding to the element fa1 are the same as the positions of the elements with different values in the vector of the simd format corresponding to the element fc1, and the vector of the simd format corresponding to the element fb1 needs to be right-rotated by 16 units relative to the vector of the simd format corresponding to the element fa1 or the element fc1. Therefore, the element fa1, the element fb1, and the element fc1 are still in the same parameter group, and the cloud server can store the position distribution vector corresponding to the element fa1 as the position distribution vector corresponding to the parameter group. For the element fb1, the cloud server can store the number of units needed to be right-rotated relative to the position distribution vector, that is, 16 units.
[0060] In addition, in the embodiments of the present specification, after the model parameters in the machine learning model are compressed and stored by using the above method, when the cloud server receives the ciphertext input data, the ciphertext input data can be converted into a vector in a preset ciphertext format, and each model parameter compressed and stored is restored into a vector in a preset ciphertext format corresponding to each model parameter, so as to process the ciphertext input data. Specifically, taking CKKS ciphertext input data as an example, after the cloud server receives the CKKS ciphertext input data, the CKKS ciphertext input data can be converted into a simd format vector as an input vector, the model parameters in the machine learning model that need to process the input vector are determined as target parameters, then the single-point parameter value of the stored target parameters and the position distribution corresponding to the target parameters are read, the simd format vector corresponding to the target parameters is restored as a parameter vector according to the parameter value of the target parameters and the position distribution corresponding to the target parameters, and finally the parameter vector is used to process the input vector.
[0061] Specifically, when the simd format vector corresponding to the target parameters is restored, the product of the parameter value of the target parameters and the position distribution vector corresponding to the target parameters can be determined as the restored simd format vector corresponding to the target parameters, as shown in Figure 7
[0062] Figure 7 The schematic diagram for restoring the simd format vector corresponding to the target parameters provided by the embodiments of the present specification is Figure 7 Figure 5 As shown in the position distribution vector in Figure 5 is a vector composed of 0 and 1, when the parameter vector corresponding to the element fa1 is restored, the product of the single-point parameter value of the stored element fa1 and the position distribution vector is determined, and the obtained vector is the restored parameter vector corresponding to the element fa1.
[0063] Figure 8 Another schematic diagram for restoring the simd format vector corresponding to the target parameters provided by the embodiments of the present specification is Figure 8 Figure 6 As shown in the position distribution vector in Figure 6 If the element fbl corresponds to a parameter vector, then, when the parameter vector corresponding to the element fbl is restored, the position distribution vector can be rotated according to the number of units required to rotate the position distribution vector to the specified direction, and the product of the single-point parameter value of the stored element fbl and the rotated position distribution vector is determined, and the obtained vector is the parameter vector corresponding to the element fbl restored.
[0064] The compression storage method described above is only used as an example of the convolution kernel in the CNN model. In essence, as long as there are multiple model parameters in the machine learning model whose positions of elements with different values are the same after being converted into a preset ciphertext format, the compression storage method described above can be used to compress and store the model parameters in the machine learning model.
[0065] The above is a data compression storage method provided by an embodiment of the present specification. Based on the same idea, the present specification also provides a corresponding device, a storage medium, and an electronic device.
[0066] Figure 9 A data compression storage device provided by an embodiment of the present specification is shown in the schematic diagram. The device comprises:
[0067] The acquisition module 901 is configured to acquire each model parameter in the machine learning model.
[0068] The determination module 902 is configured to determine, for each model parameter, the position distribution of elements with different values in the vector after the model parameter is converted into a preset ciphertext format.
[0069] The compression storage module 903 is configured to take each model parameter with the same position distribution as a same parameter group, and store, for each parameter group, the parameter values of the model parameters in the parameter group and the position distribution of elements with different values in the vector corresponding to any model parameter in the parameter group.
[0070] Optionally, the machine learning model is used to input ciphertext input data in the form of a vector in a preset ciphertext format.
[0071] The ciphertext input data comprises CKKS ciphertext input data.
[0072] The vector in the preset ciphertext format comprises a vector in simd format.
[0073] Optionally, each model parameter comprises each element in the convolution kernel in the machine learning model.
[0074] The determining module 902 is specifically configured to determine, for each element in the convolution kernel, a position of the element in the convolution kernel; determine, according to the position of the element in the convolution kernel, a position of the element in the vector after the element is converted into the vector in the preset ciphertext format; and determine, according to the position of the element in the vector, a position distribution of elements with different values in the vector.
[0075] Optionally, the determining module 902 is specifically configured to determine, after the model parameter is converted into the vector in the preset ciphertext format, a position distribution of elements with a value of 0 and elements with a value other than 0 in the vector.
[0076] Optionally, the compressed storage module 903 is specifically configured to determine, for each element with a non-zero value in the vector, a position of the element with the non-zero value in the vector; determine, according to the position of the element with the non-zero value in the vector, a position distribution vector corresponding to the element with the non-zero value, and store the position distribution vector, where the position distribution vector has the same dimension as the vector in the preset ciphertext format, and an element at a position corresponding to the element with the non-zero value in the position distribution vector has a value of 1, and elements at other positions have values of 0.
[0077] Optionally, the apparatus further includes:
[0078] The processing module 904 is configured to convert CKKS ciphertext input data into the vector in the preset ciphertext format as an input vector; determine a model parameter in the machine learning model that needs to process the input vector as a target parameter; read a parameter value of the target parameter and a position distribution corresponding to the target parameter; restore a vector in the preset ciphertext format corresponding to the target parameter as a parameter vector according to the parameter value of the target parameter and the position distribution corresponding to the target parameter; and process the input vector by using the parameter vector.
[0079] Optionally, the stored position distribution corresponding to the target parameter is a position distribution vector corresponding to the target parameter, the position distribution vector has the same dimension as the vector in the preset ciphertext format, and an element at a position corresponding to the target parameter in the position distribution vector has a value of 1, and elements at other positions have values of 0.
[0080] The processing module 904 is specifically configured to determine a product of the parameter value of the target parameter and the position distribution vector corresponding to the target element as the restored vector in the preset ciphertext format corresponding to the target parameter.
[0081] This specification also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can be used to execute the data compression storage method provided above.
[0082] based on Figure 1 The data compression storage method shown in the embodiment of this specification also provides Figure 1 The structural diagram of the electronic device shown in FIG. Figure 10 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile storage, and may also include other hardware required for its operations. The processor reads the corresponding computer program from the non-volatile storage into the memory and then runs it, implementing the aforementioned data compression storage method.
[0083] The foregoing is merely an example of the present invention and is not intended to limit the present invention. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.
Claims
1. A method for compressing and storing data, the method comprising: Get the model parameters in the machine learning model; For each model parameter, determining the position distribution of elements with different values in the vector after converting the model parameter into a vector in a preset ciphertext format; The model parameters with the same position distribution are regarded as the same parameter group, and for each parameter group, the parameter value of each model parameter in the parameter group and the position distribution of elements with different values in the vector corresponding to any model parameter in the parameter group are stored.
2. The method of claim 1, wherein the machine learning model is configured to input ciphertext input data of a vector in a preset ciphertext format; The ciphertext input data includes CKKS ciphertext input data; The vector in the preset ciphertext format includes a vector in the SIMD format.
3. The method of claim 1, wherein the model parameters include elements of a convolution kernel in the machine learning model; For each model parameter, after converting the model parameter into a vector in a preset ciphertext format, the position distribution of elements with different values in the vector is determined, specifically including: For each element in the convolution kernel, determining a position of the element in the convolution kernel; Determining, according to the position of the element in the convolution kernel, the position of the element in the vector after converting the element into the vector in the preset ciphertext format; According to the position of the element in the vector, the position distribution of elements with different values in the vector is determined.
4. The method of claim 1, wherein determining the position distribution of elements of different values in the vector after converting the model parameter into a vector in a preset ciphertext format comprises: After determining the model parameter is converted into a vector in a preset ciphertext format, the position distribution of elements with a value of 0 and elements with a value not equal to 0 in the vector is determined.
5. The method according to claim 4, storing the position distribution of elements with different values in the vector corresponding to any model parameter in the parameter group, specifically comprising: For each non-zero element in the vector, determine the position of the non-zero element in the vector; According to the position of the element with a non-zero value in the vector, the position distribution vector corresponding to the element with a non-zero value is determined and stored, wherein the position distribution vector has the same dimension as the vector in the preset ciphertext format, and the element value corresponding to the position of the element with a non-zero value in the position distribution vector is 1, and the elements at other positions are 0.
6. The method of claim 2, further comprising: Convert the CKKS ciphertext input data into a vector in the preset ciphertext format as an input vector; Determining a model parameter in the machine learning model that is required to process the input vector as a target parameter; Reading the stored parameter value of the target parameter and the position distribution corresponding to the target parameter; Restoring a vector in a preset ciphertext format corresponding to the target parameter as a parameter vector according to the parameter value of the target parameter and the position distribution corresponding to the target parameter; The input vector is processed using the parameter vector.
7. The method of claim 6, wherein the stored position distribution corresponding to the target parameter is a position distribution vector corresponding to the target parameter, the position distribution vector has the same dimension as the vector in the preset ciphertext format, and the element corresponding to the position of the target parameter in the position distribution vector has a value of 1, and the elements at other positions have a value of 0; Restoring a vector in a preset ciphertext format corresponding to the target parameter according to the parameter value of the target parameter and the position distribution corresponding to the target parameter specifically includes: The product of the parameter value of the target parameter and the position distribution vector corresponding to the target element is determined as the restored vector in the preset ciphertext format corresponding to the target parameter.
8. A data compression storage device, comprising: The acquisition module is used to obtain the model parameters in the machine learning model; a determination module configured to determine, for each model parameter, a position distribution of elements with different values in the vector after the model parameter is converted into a vector in a preset ciphertext format; The compression storage module is used to treat the model parameters with the same position distribution as the same parameter group, and for each parameter group, store the parameter value of each model parameter in the parameter group, as well as the position distribution of elements with different values in the vector corresponding to any model parameter in the parameter group.
9. A computer-readable storage medium storing a computer program, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by a processor.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program.