Federal learning model security compression method under satellite node resource isomerism and related equipment

Abstract

One or more embodiments of the invention provide a federated learning model security compression method under satellite node resource isomerism and related equipment. The method comprises the steps that a federated learning model is compressed based on a sparse matrix compression algorithm, first compression parameters of the federated learning model are obtained, and each first compression parameter corresponds to an index value; compressing and encrypting the first compression parameter and the index value respectively; wherein the first compression parameter is compressed based on a fitting algorithm and encrypted based on a homomorphic encryption algorithm. According to the method and the device, the model communication efficiency can be improved under the condition of ensuring the security of the model.

Description

Technical Field

[0001] One or more embodiments of the present application relate to the technical field of model aggregation in federated learning, and in particular to a method for securely compressing a federated learning model under heterogeneous satellite node resources and related equipment. Background Art

[0002] In recent years, with the deep integration of federated learning technology with application scenarios such as the Internet of Things, satellite computing, and command and control, client devices have evolved from a single type to a system with heterogeneous resources (computing, communication, storage, and data). In a federated learning environment with heterogeneous client satellite node resources, in order to overcome the lack of communication resources of some satellite nodes during long-distance communication, the model aggregation stage needs to achieve secure model aggregation by transmitting fewer parameters. At the same time, the client also needs to compress the sub-model parameters to reduce the amount of parameters transmitted.

[0003] Current technologies such as sparse matrix compression and differential privacy aim to achieve secure model aggregation with low communication cost. However, these solutions still have shortcomings in communication efficiency. Therefore, improving the compression rate of model compression algorithms has become a technical challenge that needs to be solved urgently. How to efficiently trim and compress models is a key issue. Summary of the invention

[0004] In view of this, the purpose of one or more embodiments of the present application is to propose a method and related equipment for secure compression of a federated learning model under heterogeneous satellite node resources to solve the problems raised by the background technology.

[0005] Based on the above objectives, one or more embodiments of the present application provide a method for securely compressing a federated learning model under heterogeneous satellite node resources, including:

[0006] Compressing the federated learning model based on a sparse matrix compression algorithm to obtain first compression parameters of the federated learning model, each of the first compression parameters corresponding to an index value;

[0007] The first compression parameter and the index value are compressed and encrypted respectively; wherein the first compression parameter is compressed based on a fitting algorithm and encrypted based on a homomorphic encryption algorithm.

[0008] Optionally, compressing and encrypting the first compression parameter includes:

[0009] Compressing the first compression parameter based on a piecewise high-order polynomial fitting algorithm to obtain a second compression parameter;

[0010]

[0011] P j (x) = a j0 +aj1 x+a j2 x 2 +...+a jk x k ;

[0012] Among them, P j (x) represents interval I j The corresponding second compression coefficient, x represents the first compression parameter, x∈A′, A′ represents the first compression parameter matrix, a0, a1, ..., a n represents the compression factor;

[0013] Encrypting the compression coefficient based on a shared public key pk and a homomorphic encryption algorithm to obtain an encrypted form of the second compression parameter; the homomorphic encryption algorithm satisfies addition homomorphism and multiplication homomorphism;

[0014]

[0015] P' j (x) = E(a j0 )+E(a j1 )x+E(a j2 )x 2 +...+E(a jk )x k .

[0016] Optionally, compressing the index value includes:

[0017] Determine an index value set V of the index value; wherein, S represents the full set of index values, n and m represent the size of the parameter matrix A of the federated learning model;

[0018] For the index value, based on the preset hash function, a corresponding hash value set H = {H1, H2, ..., H m}, where H i ={h i1 ,h i2 ,...,h iV};

[0019] Set verification array Arr = {0} n , and set the position corresponding to the hash value in the verification array to 1, To get a standard Bloom filter;

[0020] Based on the hash function, the hash value H′={H′1, H′2, ..., H′ m}, to obtain the false positive set Among them, H'i ={h' i1 ,h' i2 ,...,h' iV};

[0021] Optimizing the hash function based on an iterator and updating the hash value set until |F|<ε to obtain a learning Bloom filter;

[0022] The index value is compressed based on the learned Bloom filter.

[0023] Optionally, the compression rate of the federated learning model is adjusted by dynamically adjusting the parameters of the deep learning model in the learning Bloom filter; the compression rate of the federated learning model is dynamically adjusted based on the computing power of the terminal applied by the federated learning model and the communication distance between the terminal and the central server.

[0024] Optionally, the compression ratio is determined by the following steps:

[0025] Establish the time complexity model of compression algorithm;

[0026]

[0027] Among them, t matrix represents the time complexity of the sparse matrix compression algorithm, t matrix =(Rank(A i )) 2 / FLOPS, A i represents the parameter matrix of the i-th layer of the federated learning model, FLOPS represents the number of floating-point operations per second of the terminal, represents the compression time of the index value, FLOPs Blo represents the training time of the machine learning model in the learning Bloom filter, r Blo represents the training round of the learning model in the learning Bloom filter, t value represents the compression time of the first compression parameter, t value =(len i ) 3 / FLOPS,len i represents the array length of the parameter matrix of the i-th layer of the federated learning model after sparse matrix compression;

[0028] According to the time complexity of the compression algorithm, a total time complexity model is established;

[0029]

[0030] Among them, t commDenotes the communication time complexity, t comm =64bit·m·R comp / V comm , R comp represents the compression ratio, R comp =f(r Blo ), V comm represents the communication rate of the terminal;

[0031] According to the total time complexity model, an optimal solution for the compression rate is calculated.

[0032] Optionally, it also includes:

[0033] Initialize a feasible set M = S\F, and sort the elements in the feasible set into a feasible set sequence M′ = sort(M);

[0034] Calculate the hash value array H corresponding to the elements in the feasible set sequence based on the hash function M ;

[0035]

[0036] From the hash value array H M Filter out valid index sets

[0037]

[0038] The valid index set Mapped into a position sequence to indicate the position of the index value in the first compression parameter.

[0039] Optionally, it also includes:

[0040] Extract the position sequence from the encrypted second compressed parameter And substitute the ciphertext sequence of the matrix numerical approximation in turn To get the corresponding ciphertext;

[0041] Position-based sequence and the ciphertext sequence C to restore the homomorphic ciphertext of the parameter matrix A of the federated learning model;

[0042] The encrypted first compression parameter is decrypted based on the homomorphic ciphertext.

[0043] Based on the same inventive concept, one or more embodiments of the present application further provide a federated learning model security compression device under heterogeneous satellite node resources, including:

[0044] A first compression module is configured to compress the federated learning model based on a sparse matrix compression algorithm to obtain first compression parameters of the federated learning model, each of the first compression parameters corresponding to an index value;

[0045] The second compression module is configured to compress and encrypt the first compression parameter and the index value respectively; wherein the first compression parameter is compressed based on a fitting algorithm and encrypted based on a homomorphic encryption algorithm.

[0046] Based on the same inventive concept, one or more embodiments of the present application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, a method for securely compressing a federated learning model under heterogeneous satellite node resources as described in any one of the above items is implemented.

[0047] Based on the same inventive concept, one or more embodiments of the present application also provide a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute any of the above-mentioned methods for secure compression of federated learning models under heterogeneous satellite node resources.

[0048] From the above, it can be seen that the secure compression method of the federated learning model under heterogeneous satellite node resources provided by one or more embodiments of the present application compresses the federated learning model based on a sparse matrix compression algorithm to obtain the first compression parameter of the federated learning model, and each of the first compression parameters corresponds to an index value; the first compression parameter and the index value are compressed and encrypted respectively; wherein the first compression parameter is compressed based on a fitting algorithm and encrypted based on a homomorphic encryption algorithm.

[0049] One or more embodiments of the present application improve the compression rate of the federated learning model through secondary compression, further improving the model communication efficiency. In addition, the present application achieves a high compression rate of the model under encryption conditions, ensuring the security of the model.

[0050] The present application provides a federated learning model security compression device, electronic device and computer-readable storage medium under heterogeneous satellite node resources, which can implement the steps of the above-mentioned federated learning model security compression method under heterogeneous satellite node resources, and therefore also have the beneficial effects of the above-mentioned federated learning model security compression method under heterogeneous satellite node resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate one or more embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only one or more embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 A flowchart of a method for securely compressing a federated learning model under heterogeneous satellite node resources according to one or more embodiments of the present application;

[0053] Figure 2 This is a schematic diagram of the structure of a secure compression device for a federated learning model under heterogeneous satellite node resources according to one or more embodiments of the present application;

[0054] Figure 3 A schematic diagram of a process for secure compression of a federated learning model in one or more embodiments of the present application;

[0055] Figure 4 A schematic diagram of a process for decompressing index values ​​of one or more embodiments of the present application;

[0056] Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to one or more embodiments of the present application. DETAILED DESCRIPTION

[0057] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0058] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present application should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in one or more embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0059] like Figure 3As shown in Figure 1, the secure aggregation of federated learning models can usually be divided into four stages: model processing, transmission, decompression, and aggregation. Model processing includes model compression and model encryption, of which model compression is the main means of reducing communication costs. In the case of heterogeneous resources, different clients can have different compression rates for the model. The privacy of the client model needs to be protected during the model aggregation stage.

[0060] The technical solution of the present application is mainly applied to the processing stage of the model, and a corresponding model decompression solution is provided to improve the model compression rate while ensuring the secure transmission of the model, and further improve the efficiency of model security aggregation.

[0061] refer to Figure 1 The method for securely compressing a federated learning model under heterogeneous satellite node resources of one or more embodiments of the present application includes the following steps:

[0062] Step S101: compress the federated learning model based on a sparse matrix compression algorithm to obtain first compression parameters of the federated learning model, where each of the first compression parameters corresponds to an index value.

[0063] Step S102: compress and encrypt the first compression parameter and the index value respectively; wherein the first compression parameter is compressed based on a fitting algorithm, and encrypted based on a homomorphic encryption algorithm.

[0064] In step S101, the sparse matrix compression algorithm is mainly used to reduce the space required for storing and transmitting sparse matrices. Using sparse representation technology, data is converted into sparse form, and a higher data compression rate can be obtained. The sparse matrix compression algorithm of the present application can adopt a compressed storage form (Compressed Sparse Row, CSR), a sparse column storage form (Compressed Sparse Column, CSC), a coordinate storage format (Coordinate List, COO), a block sparse storage, hash compression, quantization and encoding, an approximate method and sparse coding, etc., and the present application does not limit the specific method adopted by the sparse matrix compression algorithm.

[0065] Through the above sparse matrix compression algorithm, a first compression parameter after the first compression can be obtained, and a first compression parameter matrix can be constructed based on the first compression parameter. The first compression parameter matrix is ​​a sparse matrix.

[0066] In addition to the first compression parameters, the sparse matrix algorithm can also obtain index values, each first compression parameter corresponds to an index value, and the index value is used to indicate the position of the first compression parameter in the first compression parameter matrix.

[0067] In step S102, the first compression parameter and the index value are compressed and encrypted respectively.

[0068] It should be noted that in this step, the first compression parameter is compressed and encrypted synchronously, which can achieve high compression of the first compression parameter in an encrypted state. The first compression parameter is a value with no typical rules, and the related art does not realize synchronous compression and encryption of such a value with no typical rules.

[0069] The specific reasons are: the distribution of irregular values ​​may not conform to any known pattern, making traditional compression algorithms (such as RLE or Huffman coding) ineffective. High compression ratios usually mean more complex algorithms, which may lead to slower compression speeds. In applications with high performance requirements, how to balance the two is a challenge. At the same time, decompression speed is equally important, especially in real-time systems, where fast decompression of data is key.

[0070] Federated learning requires the protection of the privacy and security of client data during the model aggregation phase, so the transmission of the numerical part needs to consider security, and its compression algorithm cannot damage the encryption algorithm. Commonly used compression algorithms are divided into lossy compression and lossless compression, among which lossy compression and lossless compression have significant differences in compression rate and data fidelity. Lossy compression can usually achieve a high compression rate (up to 90% or higher), reducing the file size by discarding part of the information, and is suitable for media content with relatively low quality requirements; relatively speaking, lossless compression has a lower compression rate (usually between 20% and 50%), but can ensure the integrity and accuracy of the data, and is suitable for applications such as important documents and scientific data that require all original information to be retained. In this application scenario, the accuracy requirements for the numerical part are not high, but the relevant technology considers that direct compression of the ciphertext with lossy compression will cause a huge change in the plaintext, so only a lossless compression scheme with a lower compression rate can be adopted.

[0071] The most commonly used algorithm for ensuring data security computing in this scenario is homomorphic encryption. It is complex and challenging to compress encrypted data and maintain its homomorphism. First, the compression of encrypted data requires identifying and utilizing redundancy in the data without decryption, while the characteristics of homomorphic encryption require that any operation on the data must be performed in an encrypted state, which limits the application of conventional compression algorithms because these algorithms usually rely on direct access to and analysis of plaintext data. Second, the structure of homomorphic encryption often leads to uneven distribution of data, which increases the difficulty for compression algorithms to identify patterns and redundancy. In addition, maintaining homomorphism means that the reversibility and security of operations must be ensured during the compression process.

[0072] In order to solve this technical problem, the present application designs a homomorphic encryption algorithm based on additive homomorphism and multiplication homomorphism and a piecewise polynomial fitting algorithm, uses the polynomial fitting algorithm to perform lossy compression on the numerical part, and performs homomorphic encryption on the coefficients of the high-order polynomial during compression. Among them, the properties of multiplication homomorphism and additive homomorphism are used to ensure that the numerical ciphertext predicted by the coefficient ciphertext has an error less than ε after decryption and the compressed numerical value, thereby overcoming the difficulty of compressing encrypted data, so that the result decompressed by the server using the piecewise high-order polynomial is equivalent to the homomorphic ciphertext of the original numerical approximation. And because the homomorphic ciphertexts are homomorphically additivity, the coefficient matrices decompressed during decompression still maintain homomorphic additivity, so that the aggregation of model parameters can proceed normally.

[0073] At the same time, using piecewise polynomials for numerical compression is an efficient technology, especially suitable for data sets with obvious trends or local regularities. The main advantage of this scheme is that it can significantly reduce storage space because it only needs to save the polynomial coefficients and segmentation points of each segment instead of the original data, thereby reducing the storage of redundant information. At the same time, the flexibility of the piecewise polynomial allows the selection of appropriate polynomial orders and segmentation strategies according to data characteristics, thereby achieving effective compression while maintaining high fitting accuracy. Therefore, this scheme has a better compression rate than the lossless compression algorithm of the related technology.

[0074] Specifically, in an embodiment of the present application, compressing and encrypting the first compression parameter may include:

[0075] Assume the parameter matrix of the federated learning model The matrix numerical sequence generated by the parameter matrix after the first sparse matrix compression is:

[0076] The matrix numerical sequence A′ is fitted with a piecewise polynomial, and the fitting result can be expressed as:

[0077]

[0078] Among them, P j (x) represents interval I j The corresponding second compression coefficient, x represents the first compression parameter, x∈A′, A′ represents the first compression parameter matrix, that is, the parameter matrix after compression by the sparse matrix algorithm, a0, a1, ..., a n Indicates the compression factor.

[0079] After compression, the fitted polynomial coefficients are homomorphically encrypted. In this application, a public key pk that needs to be shared with other clients is set and the coefficients in the fitting result are homomorphically encrypted using the public key to obtain the piecewise high-order polynomial in ciphertext form:

[0080]

[0081] Among them, P' j (x) = E(a j0 )+E(a j1 )x+E(a j2 )x 2 +…+E(a jk )x k .

[0082] Corresponding to the compression and encryption process, at the central server end, the decompression process may include:

[0083] Receive the above-mentioned piecewise high-order polynomial P′(x) and extract the position sequence Substitute them in turn to get the ciphertext sequence of the numerical approximation of the matrix corresponding to the model parameters:

[0084] Using position sequence The decompression can be completed by restoring the homomorphic ciphertext of the compressed matrix A with the ciphertext sequence C.

[0085] Among them, the above position sequence It can be determined based on the above index value.

[0086] This application also proposes a method for encrypting and compressing index values.

[0087] In traditional federated learning, it is usually assumed that the participating parties have similar computing resources and data distribution. However, in practical applications, different participants may have different types and numbers of computing devices, network bandwidth, and data quality and quantity. This heterogeneity of resources poses a challenge to federated learning, as traditional federated learning algorithms may not be able to handle this situation effectively.

[0088] Related technologies usually use sparse matrix compression and differential privacy technology to achieve secure model aggregation with low communication cost, but it is difficult to balance the communication and computing costs in the model aggregation stage. In other words, there is currently a lack of a method to comprehensively utilize communication resources and computing resources to evaluate the optimal compression rate, resulting in insufficient utilization of communication and computing resources of each client satellite node.

[0089] The compression algorithm used in related technologies cannot dynamically adjust the training rounds, so when setting the model training rounds, it is necessary to comprehensively consider the computing power of all terminals and set the compression rate based on the minimum computing power. However, in this case, the compression rate is generally small, and the transmission pressure of the model parameters is relatively large.

[0090] In the process of implementing the present application, the applicant found that the dynamic adjustment of the compression rate can be achieved by using a learning Bloom filter. In the learning Bloom filter, the training round determines the size of the false positive set and the shortest available verification array length. The data that needs to be transmitted during the transmission process is the verification array length, the false positive set and the hash function group, wherein the hash function group only needs to transmit m keys with a length of up to 16B when transmitting, and a hash function can be generated by a hash function-based message authentication code (Hash-Based Message Authentication Code, HMAC) algorithm when the hash function algorithm is agreed in advance, so this part takes up very little space. It can be seen that the compression rate mainly depends on the length of the verification array and the size of the false positive set, and the more training rounds, the smaller the minimum length available for the verification array and the fewer elements in the false positive set. Therefore, the compression rate of the federated learning model can be dynamically adjusted by using the learning Bloom filter compression index value, and a high compression rate can be achieved when there are more computing power resources, improving transmission efficiency; when computing power resources are scarce, you can also choose to perform fewer rounds of training to shorten the time of compressing data, and finally achieve the purpose of optimizing the total time consumption.

[0091] First, the process of implementing index value compression based on the learning Bloom filter in this application is introduced.

[0092] Implementing index value compression based on learned Bloom filters can include:

[0093] The parameter matrix of the federated learning model is The data range of the index part can be determined, and the full set of index values ​​can be expressed as The index value set obtained after the first compression is set to

[0094] For each index value in V, use the pre-given m hash functions to find m sets of hash values, set to H = {H1, H2, ..., H m}, H i ={h i1 ,h i2 ,...,h iV}, and set the verification array Arr of length n bits = {0} n . Set the position in the verification array corresponding to the hash value to 1: Thus a standard Bloom filter is generated;

[0095] Use m hash functions to hash the set V c =S\V elements to obtain hash values ​​H′={H1',H'2,...,H' m}, H i'={h i '1,h i '2,...,h i ' V}, the false positive set is obtained as:

[0096]

[0097] Use iterators to optimize the hash function group and update H, repeating multiple rounds until |F|<ε.

[0098] The data transmitted to the central server includes the verification array Arr, the false positive set F, and the hash function group. The encryption of the above data can be implemented by any encryption algorithm, and this application does not limit the selection of a specific encryption algorithm.

[0099] Correspondingly, such as Figure 4 As shown, the decompression process of the index value may include:

[0100] Initialize the feasible set to M = S\F, sort the elements in the feasible set into a feasible set sequence M′ = sort(M);

[0101] The corresponding hash value array is calculated for the feasible set sequence through each hash function in the received hash function group, which is expressed as: in

[0102] Then extract the valid index set by verifying the array:

[0103]

[0104] The extracted Remapping to the position in the sparse matrix A completes the decompression of the index part.

[0105] The learning Bloom filter can also be used to dynamically adjust the compression rate.

[0106] In this application, the applicant comprehensively utilizes the time complexity of the compression algorithm and the client satellite node resource evaluation results to measure the optimal compression rate.

[0107] Among them, the client resource indicators include: the number of floating-point operations per second FLOPS of the device, the communication rate V comm Represent the client computing and communication resources respectively. Define the client model M j The parameter matrix of the i-th layer is A i , the sparse matrix A i The length of the array after initial compression is len i .

[0108] The time cost of the compression algorithm mainly comes from sparse matrix compression, index compression and numerical compression. The time complexity of sparse matrix compression and numerical compression algorithms is relatively low, and the time cost can be set to t matrix =(Rank(A i )) 2 / FLOPS and t value =(len i ) 3 / FLOPS. The index compression time is directly related to the training and running time of the machine learning model in the learning Bloom filter. The parameters of the machine learning model in the learning Bloom filter can be adjusted to control the number of training rounds of the hash function, thereby adjusting the running time of the compression algorithm. Let the number of training rounds of the machine learning model in the learning Bloom filter be r Blo , the model calculation amount is FLOPs Blo , the time required for each round of machine learning model training is The client model transmission initialization time cost is:

[0109] Combine the amount of compressed calculations and terminal computing resources to design the terminal model compression rate R comp Functional relationship R with the number of training rounds r comp =f(r). The communication time cost t is designed by combining the data volume and the terminal communication resources. comm =64bit·m·R comp / V comm The sum of the terminal model initialization time cost and the communication time cost is taken as the total cost of the transmission process: Set the minimization of time cost as the optimization goal, perform unconstrained optimization, and find the optimal compression rate. According to the optimal compression rate, the compression algorithm is guided to obtain an optimized compression scheme that supports secure aggregation.

[0110] In the embodiments of the present application, the time complexity of the encryption algorithm may also be comprehensively considered.

[0111] It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities.

[0112] It should be noted that the method of one or more embodiments of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of one or more embodiments of the present application, and the multiple devices will interact with each other to complete the described method.

[0113] It should be noted that the above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0114] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides a federated learning model security compression device under heterogeneous satellite node resources. Figure 2 As shown, the above device comprises:

[0115] A first compression module 11 is configured to compress the federated learning model based on a sparse matrix compression algorithm to obtain first compression parameters of the federated learning model, each of which corresponds to an index value;

[0116] The second compression module 12 is configured to compress and encrypt the first compression parameter and the index value respectively; wherein the first compression parameter is compressed based on a fitting algorithm and encrypted based on a homomorphic encryption algorithm.

[0117] For the convenience of description, the above devices are described in terms of functions and modules. Of course, when implementing one or more embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0118] The apparatus of the above-mentioned embodiment is used to implement the corresponding method in the above-mentioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0119] Figure 5 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0120] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0121] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solution provided in the embodiment of the present application is implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0122] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0123] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0124] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0125] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiment of the present application, and does not necessarily include all the components shown in the figure.

[0126] The electronic device of the above embodiment is used to implement the corresponding method in the above embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0127] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0128] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0129] In addition, to simplify the description and discussion, and in order not to make one or more embodiments of the present application difficult to understand, the known power / ground connections to the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making one or more embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which one or more embodiments of the present application will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present disclosure, it is obvious to those skilled in the art that one or more embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0130] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0131] One or more embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present application should be included in the scope of protection of this disclosure.

Claims

1. A method for secure compression of a federated learning model under heterogeneous satellite node resources, characterized in that: include: Compressing the federated learning model based on a sparse matrix compression algorithm to obtain first compression parameters of the federated learning model, each of the first compression parameters corresponding to an index value; The first compression parameter and the index value are compressed and encrypted respectively; wherein the first compression parameter is compressed based on a fitting algorithm and encrypted based on a homomorphic encryption algorithm.

2. The method according to claim 1, characterized in that Compressing and encrypting the first compression parameter includes: Compressing the first compression parameter based on a piecewise high-order polynomial fitting algorithm to obtain a second compression parameter; P j (x)=a j0 +a j1 x+a j2 x 2 +…+a jk x k ; Among them, P j (x) represents interval I j The corresponding second compression coefficient, x represents the first compression parameter, x∈A′, A′ represents the first compression parameter matrix, a0, a1, ..., a n represents the compression factor; Encrypting the compression coefficient based on a shared public key pk and a homomorphic encryption algorithm to obtain an encrypted form of the second compression parameter; the homomorphic encryption algorithm satisfies addition homomorphism and multiplication homomorphism; P′ j (x)=And(a j0 )+E(a j1 )x+E(a j2 )x 2 +…+E(a jk )x k 。 3. The method according to claim 2, characterized in that The index value is compressed, including: Determine an index value set V of the index value; wherein, S represents the full set of index values, n and m represent the size of the parameter matrix A of the federated learning model; For the index value, based on the preset hash function, a corresponding hash value set H = {H1, H2, ..., H m }, where H i ={h i1 ,h i2 ,...,h iV }; Set verification array Arr = {0} n , and set the position corresponding to the hash value in the verification array to 1, To get a standard Bloom filter; Based on the hash function, the hash value H′={H′1, H′2, ..., H′ m }, to obtain the false positive set Among them, H′ i ={h′ i1 ,h′ i2 ,...,h′ iV }; Optimizing the hash function based on an iterator and updating the hash value set until |F|<ε to obtain a learning Bloom filter; The index value is compressed based on the learned Bloom filter.

4. The method according to claim 3, characterized in that The compression rate of the federated learning model is adjusted by dynamically adjusting the parameters of the deep learning model in the learning Bloom filter; the compression rate of the federated learning model is dynamically adjusted based on the computing power of the terminal applied by the federated learning model and the communication distance between the terminal and the central server.

5. The method according to claim 4, characterized in that The compression ratio is determined by the following steps: Establish the time complexity model of compression algorithm; Among them, t matrix represents the time complexity of the sparse matrix compression algorithm, t matrix =(Rank(A i )) 2 / FLOPS, A i represents the parameter matrix of the i-th layer of the federated learning model, FLOPS represents the number of floating-point operations per second of the terminal, represents the compression time of the index value, FLOPs Blo represents the training time of the machine learning model in the learning Bloom filter, r Blo represents the training round of the learning model in the learning Bloom filter, t value represents the compression time of the first compression parameter, t value =(len i ) 3 / FLOPS,len i represents the array length of the parameter matrix of the i-th layer of the federated learning model after sparse matrix compression; According to the time complexity of the compression algorithm, a total time complexity model is established; Among them, t comm Denotes the communication time complexity, t comm =64bit·m·R comp / V comm , R comp represents the compression ratio, R comp =f(r Blo ), V comm represents the communication rate of the terminal; According to the total time complexity model, an optimal solution for the compression rate is calculated.

6. The method according to claim 3, characterized in that Also includes: Initialize a feasible set M = S\F, and sort the elements in the feasible set into a feasible set sequence M′ = sort(M); Calculate the hash value array H corresponding to the elements in the feasible set sequence based on the hash function M ; From the hash value array H M Filter out valid index sets The valid index set Mapped into a position sequence to indicate the position of the index value in the first compression parameter.

7. The method according to claim 6, characterized in that Also includes: Extract the position sequence from the encrypted second compressed parameter And substitute the ciphertext sequence of the matrix numerical approximation in turn To get the corresponding ciphertext; Position-based sequence and the ciphertext sequence C to restore the homomorphic ciphertext of the parameter matrix A of the federated learning model; The encrypted first compression parameter is decrypted based on the homomorphic ciphertext.

8. A secure compression device for a federated learning model under heterogeneous satellite node resources, characterized in that: include: A first compression module is configured to compress the federated learning model based on a sparse matrix compression algorithm to obtain first compression parameters of the federated learning model, each of the first compression parameters corresponding to an index value; The second compression module is configured to compress and encrypt the first compression parameter and the index value respectively; wherein the first compression parameter is compressed based on a fitting algorithm and encrypted based on a homomorphic encryption algorithm.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute any one of claims 1 to 7.

Images