Information transmission method, device, equipment and computer readable storage medium

Abstract

The present invention discloses an information transmission method, device, equipment and computer-readable storage medium, the method comprising: obtaining information to be transmitted in a federated learning task; performing approximate quantization processing on the information to be transmitted to obtain approximate quantization information of the information to be transmitted; and sending the approximate quantization information to a receiver, so that the receiver can continue to perform the federated learning task based on the approximate quantization information. The present invention improves the communication efficiency in the federated learning process, reduces the amount of calculation in the federated learning process, and improves the efficiency of federated learning modeling.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an information transmission method, device, equipment and computer-readable storage medium. Background Art

[0002] With the development of computer technology, more and more technologies (big data, distributed, blockchain, artificial intelligence, etc.) are being applied in the financial field. The traditional financial industry is gradually transforming to financial technology (Fintech). However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology.

[0003] At present, the federated learning modeling method of jointly training models with multiple parties without leaving the local data has been applied to more and more fields to solve the problem of "data islands". At present, in the process of federated learning, each party will send gradient information or model parameter information to each other, and through the transmission of this information, collaborate to complete the federated learning task. However, when the gradient information or model parameter information is sent directly, since the gradient information and model parameter information are generally high-dimensional tensors, for example, each element is a 32- or 16-bit floating point precision decimal, the required storage space is very large, which will increase the network communication cost during transmission, and also make the information transmission efficiency low, and the federated learning modeling efficiency is low. At present, there is a solution to quantize the gradient information in the form of a transformation function to reduce the storage space occupied by the gradient information, thereby improving the communication efficiency. However, this solution of quantization through a transformation function not only has a large amount of calculation, but also requires the receiver to restore the quantized gradient information, which further increases the amount of calculation, thereby making the federated learning modeling efficiency low. Summary of the invention

[0004] The main purpose of the present invention is to provide an information transmission method, device, equipment and computer-readable storage medium, aiming to improve the communication efficiency in the federated learning process while reducing the amount of calculation in the communication process to improve the efficiency of federated learning modeling.

[0005] To achieve the above object, the present invention provides an information transmission method, which comprises the following steps:

[0006] Get the information to be transmitted in the federated learning task;

[0007] Performing approximate quantization processing on the information to be transmitted to obtain approximate quantization information of the information to be transmitted;

[0008] The approximate quantization information is sent to a recipient, so that the recipient continues to perform the federated learning task based on the approximate quantization information.

[0009] Optionally, the information to be transmitted is a vector to be transmitted, and the step of performing approximate quantization processing on the information to be transmitted to obtain approximate quantized information of the information to be transmitted includes:

[0010] The vector to be transferred is approximated and quantized according to the cosine similarity formula to obtain an approximate integer vector of the vector to be transferred, and the approximate integer vector is used as the approximate quantization information.

[0011] Optionally, the step of performing approximate quantization processing on the vector to be transferred according to the cosine similarity formula to obtain an approximate integer vector of the vector to be transferred includes:

[0012] Obtaining a descending vector corresponding to the vector to be transferred;

[0013] Substituting the descending vector and the candidate quantity value into a maximization formula for maximizing the target value for calculation, selecting the candidate quantity value that maximizes the target value as the target quantity value, wherein the maximization formula is constructed based on the cosine similarity formula, in which the target value is equal to the result obtained by summing the target elements in the descending vector and dividing by the target quantity under the square root, and the target element is an element in the descending vector whose sequence number is less than or equal to the candidate quantity value;

[0014] A quantization value allocation operation is performed based on the target quantity value to obtain an approximate integer vector corresponding to the vector to be transferred.

[0015] Optionally, the step of obtaining a descending vector corresponding to the vector to be transferred includes:

[0016] Performing unitization processing on the vector to be transferred to obtain a unitized vector;

[0017] Taking the absolute value of each element in the normalized vector and arranging them in descending order to obtain a descending vector;

[0018] The step of performing a quantization value allocation operation based on the target quantity value to obtain an approximate integer vector corresponding to the vector to be transferred includes:

[0019] Determine the elements in the descending vector whose sequence numbers are less than or equal to the target quantity value as descending elements, and use the elements in the normalized elements corresponding to the descending elements as retained elements;

[0020] Assign a quantized value to each element in the normalized vector to obtain an approximate integer vector corresponding to the vector to be transmitted, wherein positive 1 is assigned to the elements greater than zero in the retained elements, negative 1 is assigned to the elements less than zero in the retained elements, and 0 is assigned to the elements other than the retained elements in the normalized vector.

[0021] Optionally, the step of performing approximate quantization processing on the vector to be transferred according to the cosine similarity formula to obtain an approximate integer vector of the vector to be transferred includes:

[0022] Substituting a preset integer vector and the vector to be transferred into a cosine similarity formula, wherein the value of each element of the preset integer vector is to be determined;

[0023] Solving the cosine similarity formula to obtain the values ​​of each element of the preset integer vector, and using the preset integer vector with the determined values ​​of each element as the approximate integer vector of the vector to be transmitted, wherein the preset integer vector with the determined values ​​of each element maximizes the cosine value of the cosine similarity formula. Optionally, after performing approximate quantization processing on the vector to be transmitted according to the cosine similarity formula to obtain the approximate integer vector of the vector to be transmitted, and using the approximate integer vector as the approximate quantization information, the method further includes:

[0024] Projecting the vector to be transferred onto the approximate integer vector to obtain a scaling factor of the approximate integer vector relative to the vector to be transferred;

[0025] The scaling factor is sent to the receiver, so that after receiving the approximate integer vector, the receiver scales the approximate integer vector based on the scaling factor and continues to perform the federated learning task based on the scaling result.

[0026] Optionally, the scaling factor includes a positive scaling factor and a negative scaling factor, and the step of projecting the vector to be transferred onto the approximate integer vector to obtain the scaling factor of the approximate integer vector relative to the vector to be transferred includes:

[0027] Extracting a positive element vector and a negative element vector from the vector to be transferred, and extracting a positive integer vector and a negative integer vector from the approximate integer vector;

[0028] Projecting the positive element vector onto the positive integer vector to obtain the positive scaling factor, and projecting the negative element vector onto the negative integer vector to obtain the negative scaling factor;

[0029] The step of sending the scaling factor to the receiving party so that the receiving party scales the approximate integer vector based on the scaling factor after receiving the approximate integer vector, so as to continue to perform the federated learning task based on the scaling result includes:

[0030] The positive scaling factor and the negative scaling factor are sent to the receiver, so that after receiving the approximate integer vector, the receiver scales the positive elements in the approximate integer vector based on the positive scaling factor and scales the negative elements in the approximate integer vector based on the negative scaling factor, so as to scale the approximate integer vector and continue to perform the federated learning task based on the scaling result.

[0031] Optionally, the step of sending the approximate quantization information to a receiver so that the receiver continues to perform the federated learning task based on the approximate quantization information includes:

[0032] The approximate quantization information is Huffman encoded, and the encoded approximate quantization information is sent to a receiver, so that the receiver can decode the encoded approximate quantization information and continue to perform the federated learning task based on the decoding result.

[0033] Optionally, the step of sending the approximate quantization information to a receiver so that the receiver continues to perform the federated learning task based on the approximate quantization information includes:

[0034] Transforming the approximate quantization information according to a preset transformation rule to obtain transformation information;

[0035] The transformation information and the transformation rule are sent to a receiver, so that the receiver can restore the transformation information based on the transformation rule to obtain the approximate quantization information, and continue to perform the federated learning task based on the approximate quantization information.

[0036] To achieve the above object, the present invention further provides an information transmission device, which is deployed on a device participating in federated learning, and includes:

[0037] The acquisition module is used to obtain the information to be transmitted in the federated learning task;

[0038] A processing module, used for performing approximate quantization processing on the information to be transmitted to obtain approximate quantization information of the information to be transmitted;

[0039] A sending module is used to send the approximate quantization information to a receiving party, so that the receiving party can continue to perform the federated learning task based on the approximate quantization information.

[0040] To achieve the above-mentioned purpose, the present invention also provides an information transmission device, which includes: a memory, a processor, and an information transmission program stored in the memory and executable on the processor, and the information transmission program implements the steps of the above-mentioned information transmission method when executed by the processor.

[0041] In addition, to achieve the above-mentioned purpose, the present invention also proposes a computer-readable storage medium, on which an information transmission program is stored, and when the information transmission program is executed by a processor, the steps of the above-mentioned information transmission method are implemented.

[0042] In the present invention, by obtaining the information to be transmitted in the federated learning task, and performing approximate quantization processing on the information to be transmitted, the approximate quantization information of the information to be transmitted is obtained, and the approximate quantization information is sent to the receiver, so that the receiver can continue to perform the subsequent federated learning task according to the approximate quantization information. In the present invention, by performing quantization processing on the information to be transmitted, the approximate quantization information is obtained, and the approximate quantization information is sent to the receiver. Since the approximate quantization information is a kind of quantization information of the information to be transmitted, the storage space occupied is much smaller than the original information to be transmitted, so that the communication overhead required for information transmission is reduced, and the communication efficiency in the federated learning process is improved. In addition, since the present invention performs approximate quantization processing on the information to be transmitted, the quantization information that is approximate to the information to be transmitted is obtained, so that after the approximate quantization information is sent to the receiver, the receiver can directly use the approximate quantization information for subsequent federated learning, and there is no need to restore the approximate quantization information, thereby reducing the calculation amount of the receiver, and also reducing the storage space occupied by the receiver, and improving the efficiency of federated learning modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the structure of the hardware operating environment involved in the embodiment of the present invention;

[0044] Figure 2 It is a flowchart of the first embodiment of the information transmission method of the present invention;

[0045] Figure 3 is a formula (1) involved in the embodiment of the present invention;

[0046] Figure 4 is a formula (2) involved in the embodiment of the present invention;

[0047] Figure 5 is a formula (3) involved in the embodiment of the present invention;

[0048] Figure 6 It is a functional schematic module diagram of a preferred embodiment of the information transmission device of the present invention.

[0049] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0050] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0051] like Figure 1 As shown, Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0052] It should be noted that the information transmission device in the embodiment of the present invention can be a smart phone, a personal computer, a server and other devices, and no specific limitation is made here.

[0053] like Figure 1 As shown, the information transmission device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0054] Those skilled in the art will understand that Figure 1 The device structure shown in the figure does not constitute a limitation on the information transmission device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0055] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module and an information transfer program. The operating system is a program that manages and controls the hardware and software resources of the device and supports the operation of the information transfer program and other software or programs.

[0056] exist Figure 1 In the device shown, the user interface 1003 is mainly used for data communication with the client; the network interface 1004 is mainly used for establishing a communication connection with the server; and the processor 1001 can be used to call the information transmission program stored in the memory 1005 and perform the following operations:

[0057] Get the information to be transmitted in the federated learning task;

[0058] Performing approximate quantization processing on the information to be transmitted to obtain approximate quantization information of the information to be transmitted;

[0059] The approximate quantization information is sent to a recipient, so that the recipient continues to perform the federated learning task based on the approximate quantization information.

[0060] Furthermore, the information to be transmitted is a vector to be transmitted, and the step of performing approximate quantization processing on the information to be transmitted to obtain approximate quantized information of the information to be transmitted includes:

[0061] The vector to be transferred is approximated and quantized according to the cosine similarity formula to obtain an approximate integer vector of the vector to be transferred, and the approximate integer vector is used as the approximate quantization information.

[0062] Furthermore, the step of performing approximate quantization processing on the vector to be transferred according to the cosine similarity formula to obtain an approximate integer vector of the vector to be transferred includes:

[0063] Obtaining a descending vector corresponding to the vector to be transferred;

[0064] Substituting the descending vector and the candidate quantity value into a maximization formula for maximizing the target value for calculation, selecting the candidate quantity value that maximizes the target value as the target quantity value, wherein the maximization formula is constructed based on the cosine similarity formula, in which the target value is equal to the result obtained by summing the target elements in the descending vector and dividing by the target quantity under the square root, and the target element is an element in the descending vector whose sequence number is less than or equal to the candidate quantity value;

[0065] A quantization value allocation operation is performed based on the target quantity value to obtain an approximate integer vector corresponding to the vector to be transferred.

[0066] Furthermore, the step of obtaining the descending vector corresponding to the vector to be transferred includes:

[0067] Performing unitization processing on the vector to be transferred to obtain a unitized vector;

[0068] Taking the absolute value of each element in the normalized vector and arranging them in descending order to obtain a descending vector;

[0069] The step of performing a quantization value allocation operation based on the target quantity value to obtain an approximate integer vector corresponding to the vector to be transferred includes:

[0070] Determine the elements in the descending vector whose sequence numbers are less than or equal to the target quantity value as descending elements, and use the elements in the normalized elements corresponding to the descending elements as retained elements;

[0071] Assign a quantized value to each element in the normalized vector to obtain an approximate integer vector corresponding to the vector to be transmitted, wherein positive 1 is assigned to the elements greater than zero in the retained elements, negative 1 is assigned to the elements less than zero in the retained elements, and 0 is assigned to the elements other than the retained elements in the normalized vector.

[0072] Furthermore, the step of performing approximate quantization processing on the vector to be transferred according to the cosine similarity formula to obtain an approximate integer vector of the vector to be transferred includes:

[0073] Substituting a preset integer vector and the vector to be transferred into a cosine similarity formula, wherein the value of each element of the preset integer vector is to be determined;

[0074] The cosine similarity formula is solved to obtain the values ​​of each element of the preset integer vector, and the preset integer vector with the determined values ​​of each element is used as the approximate integer vector of the vector to be transmitted, wherein the preset integer vector with the determined values ​​of each element maximizes the cosine value of the cosine similarity formula.

[0075] Further, after the step of performing approximate quantization processing on the vector to be transferred according to the cosine similarity formula to obtain an approximate integer vector of the vector to be transferred, and using the approximate integer vector as the approximate quantization information, the processor 1001 may also be used to call the information transfer program stored in the memory 1005 to perform the following operations:

[0076] Projecting the vector to be transferred onto the approximate integer vector to obtain a scaling factor of the approximate integer vector relative to the vector to be transferred;

[0077] The scaling factor is sent to the receiver, so that after receiving the approximate integer vector, the receiver scales the approximate integer vector based on the scaling factor and continues to perform the federated learning task based on the scaling result.

[0078] Furthermore, the scaling factor includes a positive scaling factor and a negative scaling factor, and the step of projecting the vector to be transferred onto the approximate integer vector to obtain the scaling factor of the approximate integer vector relative to the vector to be transferred includes:

[0079] Extracting a positive element vector and a negative element vector from the vector to be transferred, and extracting a positive integer vector and a negative integer vector from the approximate integer vector;

[0080] Projecting the positive element vector onto the positive integer vector to obtain the positive scaling factor, and projecting the negative element vector onto the negative integer vector to obtain the negative scaling factor;

[0081] The step of sending the scaling factor to the receiving party so that the receiving party scales the approximate integer vector based on the scaling factor after receiving the approximate integer vector, so as to continue to perform the federated learning task based on the scaling result includes:

[0082] The positive scaling factor and the negative scaling factor are sent to the receiver, so that after receiving the approximate integer vector, the receiver scales the positive elements in the approximate integer vector based on the positive scaling factor and scales the negative elements in the approximate integer vector based on the negative scaling factor, so as to scale the approximate integer vector and continue to perform the federated learning task based on the scaling result.

[0083] Furthermore, the step of sending the approximate quantization information to a receiver so that the receiver continues to perform the federated learning task based on the approximate quantization information includes:

[0084] The approximate quantization information is Huffman encoded, and the encoded approximate quantization information is sent to a receiver, so that the receiver can decode the encoded approximate quantization information and continue to perform the federated learning task based on the decoding result.

[0085] Furthermore, the step of sending the approximate quantization information to a receiver so that the receiver continues to perform the federated learning task based on the approximate quantization information includes:

[0086] Transforming the approximate quantization information according to a preset transformation rule to obtain transformation information;

[0087] The transformation information and the transformation rule are sent to a receiver, so that the receiver can restore the transformation information based on the transformation rule to obtain the approximate quantization information, and continue to perform the federated learning task based on the approximate quantization information.

[0088] Based on the above structure, various embodiments of the information transmission method are proposed.

[0089] Reference Figure 2 , Figure 2 It is a flowchart of the first embodiment of the information transmission method of the present invention.

[0090] The embodiments of the present invention provide embodiments of the information transmission method. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than here. The execution subject of each embodiment of the information transmission method of the present invention may be a device such as a smart phone, a personal computer, and a server. The device may be a device participating in federated learning. For ease of description, the following embodiments are described with the sender as the execution subject. In this embodiment, the information transmission method includes:

[0091] Step S10, obtaining information to be transmitted in the federated learning task;

[0092] During the federated learning process, the participating devices and the coordinating device may send gradient information or model parameter information to each other. Among them, the model parameter information is the parameter information of the model to be trained during the training process, such as the weight information in the neural network model, and the gradient information is the information used to update the model parameters. For example, in the horizontal federated learning process, each participating device uses its own local training data to perform local training on the local model to obtain local gradient information; each participating device sends its own local gradient information to the coordinating device, and the coordinating device fuses each local gradient information. The fusion operation can be a weighted average of each gradient information to obtain global gradient information; the coordinating device sends the global gradient information to each participating device, and each participating device uses the global gradient information to update the local model, and performs local training again based on the updated model, and repeats the cycle until the trained model meets the performance requirements, and stops training to complete the federated learning task. Similarly, each participating device can also send the local model parameter information obtained by local training of the local model to the coordinating device, and the coordinating device fuses each local model parameter information to obtain the global model parameter information, and sends the global model parameter information to each participating device, so that each participating device can continue to perform local training based on the global model parameter information.

[0093] In the above-mentioned federated learning process, in order to complete the federated learning task, both the participating device and the coordinating device may act as a sender to send gradient information or model parameter information to each other, and correspondingly, both may act as a receiver to receive the gradient information or model parameter information sent by the other party. In the process of transmitting gradient information or model parameter information, due to the large amount of data of gradient information and model parameter information, a large communication overhead is required, resulting in low communication efficiency, which affects the efficiency of federated learning modeling. Therefore, in an embodiment of the present invention, an information transmission method is proposed to solve the problem of low communication efficiency in the federated learning process.

[0094] Specifically, the sender can obtain the information to be transmitted in the federated learning task, where the information to be transmitted can be gradient information or model parameter information. For example, when the sender is a participating device and has calculated local gradient information, it needs to send it to the coordinating device (receiver) for fusion. When the coordinating device performs fusion, the sender can obtain the local gradient information and use the local gradient information as the information to be transmitted.

[0095] Step S20, performing approximate quantization processing on the information to be transmitted to obtain approximate quantization information of the information to be transmitted;

[0096] After obtaining the information to be transmitted, the sender may perform approximate quantization processing on the information to be transmitted to obtain approximate quantization information of the information to be transmitted. The approximate quantization processing process may be to retain the key information of the information to be transmitted when quantizing the information to be transmitted, so that the obtained approximate quantization information is similar to the information to be transmitted in terms of key information. Specifically, the approximate quantization processing process may be to calculate the approximate quantization information whose similarity with the information to be transmitted satisfies the preset similarity condition through a similarity algorithm. The sender may calculate the quantization information similar to the information to be transmitted based on a similarity algorithm to obtain the approximate quantization information of the information to be transmitted, and the similarity between the approximate quantization information and the information to be transmitted satisfies the preset similarity condition. The similarity algorithm may adopt some existing similarity algorithms, such as the cosine similarity formula. The preset similarity condition may be a pre-set condition, for example, the preset similarity condition may be that the similarity between the approximate quantization information and the information to be transmitted is greater than a certain threshold value, and when it is greater than the threshold value, it indicates that the approximate quantization information is very similar to the information to be transmitted. It should be noted that each information in the approximate quantized information can be information in a quantized form. For example, if each information in the original information to be transmitted is a continuous floating point number, then each information in the approximate quantized information is an integer value, which greatly reduces the storage space occupied compared to the original floating point number, thereby reducing communication overhead and improving communication efficiency.

[0097] Step S30: Send the approximate quantization information to a recipient, so that the recipient can continue to perform the federated learning task based on the approximate quantization information.

[0098] After the sender calculates the approximate quantization information of the vector to be transmitted, the sender can send the approximate quantization information to the receiver. After receiving the approximate quantization information, the receiver can continue to perform subsequent federated learning tasks based on the approximate quantization information. For example, when the receiver is a coordinating device, and the received approximate quantization information is the approximate quantization information of the local model parameter information sent by the participating devices, the receiver can fuse the various approximate quantization information to obtain the global model parameter information, and then use the global model parameter information as the information to be transmitted. It should be noted that since the approximate quantization information transmitted by the sender is information that is similar to the information to be transmitted, the receiver can directly use the approximate quantization information for subsequent operations, that is, there is no need to restore the approximate quantization information, thereby reducing the receiver's calculation amount, thereby speeding up the communication efficiency in the federated learning process, and reducing the calculation amount in the federated learning process, thereby improving the modeling efficiency of federated learning.

[0099] In this embodiment, by obtaining the information to be transmitted in the federated learning task, and performing approximate quantization processing on the information to be transmitted, the approximate quantization information of the information to be transmitted is obtained, and the approximate quantization information is sent to the receiving party, so that the receiving party can continue to perform subsequent federated learning tasks according to the approximate quantization information. In this embodiment, by performing quantization processing on the information to be transmitted, the approximate quantization information is obtained, and the approximate quantization information is sent to the receiving party. Since the approximate quantization information is a kind of quantization information of the information to be transmitted, the storage space occupied is much smaller than the original information to be transmitted, so that the communication overhead required for information transmission is reduced, and the communication efficiency in the federated learning process is improved. In addition, since the present invention performs approximate quantization processing on the information to be transmitted, the quantization information that is approximate to the information to be transmitted is obtained, so that after the approximate quantization information is sent to the receiving party, the receiving party can directly use the approximate quantization information for subsequent federated learning, and there is no need to restore the approximate quantization information, thereby reducing the amount of calculation of the receiving party, and also reducing the storage space occupied by the receiving party, and improving the efficiency of federated learning modeling. In addition, in this embodiment, by sending the approximate quantized information of the information to be transmitted to the recipient instead of sending the information to be transmitted itself to the recipient, even if the information is intercepted by a third-party attacker during the information transmission process, what is obtained is not the original information to be transmitted, and thus the input data of the federated learning model cannot be derived, and the privacy information cannot be obtained, thereby improving the privacy protection in the process of federated learning information transmission.

[0100] Further, based on the above first embodiment, a second embodiment of the information transmission method of the present invention is proposed. In this embodiment, the information to be transmitted is a vector to be transmitted, and the vector to be transmitted is a gradient vector or a parameter vector. The step S20 includes:

[0101] Step S201 , performing approximate quantization processing on the vector to be transferred according to the cosine similarity formula to obtain an approximate integer vector of the vector to be transferred, and using the approximate integer vector as the approximate quantization information.

[0102] Furthermore, the gradient information or model parameter information to be transmitted in the federated learning task may be in vector form, and therefore, the sender may obtain the gradient vector or parameter vector to be transmitted as the vector to be transmitted.

[0103] After obtaining the vector to be transmitted, the sender can perform approximate quantization processing on the vector to be transmitted according to the cosine similarity formula to obtain an approximate integer vector of the vector to be transmitted, and send the approximate integer vector as approximate quantization information to the receiver. The sender can calculate the approximate integer vector of the vector to be transmitted according to the cosine similarity formula. The principle is to find an approximate integer vector with as high similarity as possible with the vector to be transmitted. Specifically, the sender can calculate an approximate integer vector based on the cosine similarity formula so that the similarity between the approximate integer vector and the vector to be transmitted meets the preset similarity condition. The preset similarity condition can be a pre-set condition so that when the condition is met, the similarity between the approximate integer vector and the vector to be transmitted is high. For example, the preset similarity condition can be: after calculating multiple candidate approximate integer vectors, the approximate integer vector finally determined must be the one with the highest similarity to the vector to be transmitted among the multiple candidate approximate integer vectors.

[0104] It should be noted that if the gradient information or model parameter information is in the form of a tensor, the sender can expand the tensor in the channel direction to obtain multiple matrices. For each of the matrices, the rows of the matrix can be taken out for vector concatenation to obtain a high-dimensional vector. Then, multiple matrices are expanded to obtain multiple high-dimensional vectors, which are used as vectors to be transmitted. Correspondingly, after receiving the approximate integer vector corresponding to the information to be transmitted, the receiver can convert the approximate integer vector into a matrix form, for example, converting a 1*MN approximate integer vector into an M*N dimensional matrix; then combine multiple matrices into a tensor, and then perform subsequent federated learning tasks based on the tensor.

[0105] In this embodiment, the approximate integer vector of the vector to be transferred is obtained by performing approximate quantization processing on the vector to be transferred according to the cosine similarity formula, so that the approximate integer vector is as similar as possible to the vector to be transferred in direction, so that the approximate integer vector obtained after quantizing the vector to be transferred is as small as possible from the original vector to be transferred, so that when the receiver receives the approximate integer vector, it is not necessary to restore the approximate integer vector, and subsequent calculations can be performed directly on the basis of the approximate integer vector, thereby improving the communication efficiency of federated learning while reducing the amount of calculation of the receiver, thereby reducing the amount of calculation in the federated learning process, and improving the modeling efficiency of federated learning. In addition, by making the approximate integer vector obtained after quantizing the vector to be transferred as small as possible from the original vector to be transferred, it is achieved that while improving the privacy protection effect and improving the information dissemination efficiency in the federated learning process, the information transmitted can be minimized, and the model in the federated learning can be converged.

[0106] Furthermore, the step of performing approximate quantization processing on the vector to be transferred according to the cosine similarity formula in step S201 to obtain an approximate integer vector of the vector to be transferred includes:

[0107] Step S2011, substituting a preset integer vector and the vector to be transferred into a cosine similarity formula, wherein the value of each element of the preset integer vector is to be determined;

[0108] Step S2012, solving the cosine similarity formula to obtain the values ​​of each element of the preset integer vector, and using the preset integer vector with the determined values ​​of each element as the approximate integer vector of the vector to be transmitted, wherein the preset integer vector with the determined values ​​of each element maximizes the cosine value of the cosine similarity formula.

[0109] Further, the process of the sender calculating the approximate integer vector based on the cosine similarity formula can be specifically as follows: substituting the preset integer vector and the vector to be transmitted into the cosine similarity formula, that is, calculating the cosine value of the preset integer vector and the vector to be transmitted, wherein the value of each element of the preset integer vector is to be determined, and specifically, each element value can be randomly initialized; solving the value of each element of the preset integer vector based on the cosine similarity formula, and solving each element value so that the cosine value of the cosine similarity formula is maximized, that is, the cosine value of the preset integer vector and the vector to be transmitted is maximized; using the preset integer vector with each element value determined as the approximate integer vector of the vector to be transmitted. The solving process can be: selecting multiple possible element values ​​of the preset integer vector, calculating the corresponding cosine values ​​according to the multiple possible element values, selecting the largest cosine value from the multiple cosine values, and using each element value corresponding to the cosine value as the element value finally determined by the preset integer vector. It can be understood that when the cosine value of two vectors is the largest, the two vectors are closest in direction.

[0110] Furthermore, in one embodiment, a method for calculating an approximate integer vector based on a cosine similarity formula is provided, which makes the calculation efficiency faster, thereby saving calculation resources and improving the overall communication efficiency. Specifically, the step of performing approximate quantization processing on the vector to be transmitted according to the cosine similarity formula in step S201 to obtain an approximate integer vector of the vector to be transmitted includes:

[0111] Step S2013, obtaining a descending vector corresponding to the vector to be transferred;

[0112] The descending vector corresponding to the vector to be transferred can be obtained by calculating the absolute value of each element value of the vector to be transferred and then arranging them in descending order to obtain the descending vector corresponding to the vector to be transferred. Each element value of the obtained descending vector is greater than zero and is arranged from large to small according to the value.

[0113] Furthermore, obtaining the descending vector corresponding to the vector to be transferred may also be: normalizing the vector to be transferred to obtain a normalized vector; taking the absolute value of each element in the normalized vector and arranging them in descending order to obtain a descending vector. The normalization process is to keep the direction of the vector to be transferred unchanged and to convert the length of the vector to be transferred to 1. The specific process can refer to the existing process of normalizing a vector, and will not be described in detail here.

[0114] Step S2014, substituting the descending vector and the candidate quantity value into a maximization formula for maximizing the target value for calculation, and selecting the candidate quantity value that maximizes the target value as the target quantity value, wherein the maximization formula is constructed based on the cosine similarity formula, and in the maximization formula, the target value is equal to the result obtained by summing the target elements in the descending vector and dividing by the target quantity under the square root, and the target element is an element in the descending vector with a sequence number less than or equal to the candidate quantity value;

[0115] A maximization formula can be constructed based on the cosine similarity formula. The maximization formula is a formula for maximizing the target value. In the formula, the target value is equal to the result obtained by summing the target elements in the descending vector and dividing it by the target quantity under the square root, wherein the target element is an element in the descending vector whose sequence number is less than or equal to the candidate quantity value. The process of solving the maximization formula is to substitute each candidate quantity value and the target element in the descending vector into the maximization formula to obtain the candidate quantity value that maximizes the target value.

[0116] Step S2015, performing a quantization value allocation operation based on the target quantity value to obtain an approximate integer vector corresponding to the vector to be transferred.

[0117] After the target quantity value is calculated, the operation of assigning the quantization value can be performed based on the target quantity value, that is, the values ​​of each element of the approximate integer vector are determined, and then the approximate integer vector corresponding to the vector to be transferred is obtained. Specifically, the number of elements of the approximate integer vector is the same as the number of elements of the normalized vector, and the sequence number of the elements in the approximate integer vector corresponds to the sequence number of the elements in the normalized vector. Then, a quantization value can be assigned to each element of the normalized vector, and the quantization value is used as the element value of the corresponding position of the approximate integer vector, so as to obtain the approximate integer vector.

[0118] Furthermore, the elements in the descending vector whose serial numbers are less than or equal to the target quantity value can be determined as descending elements, that is, the first several largest elements are used as descending elements. The descending vector is a vector sorted in descending order after the absolute value of the normalized vector is calculated. Then each element in the normalized vector corresponds to each element in the descending vector. The elements corresponding to each descending element in the normalized vector can be used as reserved elements, that is, the several elements with the largest absolute values ​​in the normalized vector are used as reserved elements. Further, 0 is assigned to the elements other than the reserved elements in the normalized vector, positive 1 is assigned to the elements greater than zero in the reserved elements, negative 1 is assigned to the elements less than zero in the reserved elements, and if the reserved element is 0, 0 is assigned accordingly. Then, the quantization value is assigned to each element of the normalized vector, and an approximate integer vector is obtained. The value of each element in the approximate integer vector is 0, -1 or +1.

[0119] The principle is: using W = (w1, w2, ..., w n ) to represent the vector to be transferred, W 0 =(a1, a2, ..., a n ) is the vector obtained by normalizing W, and the approximate integer vector T = (t1, t2, ..., t i ,…t n ), where t i ∈{0, ±1} or t i ∈{±1}, according to the cosine similarity formula, establish Figure 3 W, W shown 0 and T. According to the relationship (1), to maximize the cosine value, the denominator of the rightmost term in the relationship needs to be larger and the numerator smaller. The numerator is the square root of the sum of the squares of the values ​​of each element in T, which can be converted to the square root of the number of elements in T whose absolute value is 1. The denominator is W 0 The elements of T are multiplied and summed. In order to make the denominator larger, the absolute value of the elements in T is 1, and it is consistent with W. 0 When the corresponding elements in T have the same sign, the denominator will be larger, and when there are more elements with an absolute value of 1 in T, the numerator will also become larger. Then, we need to find the appropriate number of elements with an absolute value of 1 in T to maximize the result, and when there are more elements with an absolute value of 1 in T, the corresponding W 0 The larger the element in is, the larger the denominator is, so W 0 The elements in T corresponding to the largest elements in are assigned quantized values ​​with an absolute value of 1.

[0120] According to the above reasoning process, equation (1) is transformed into Figure 4 The formula (2) shown in the figure is the maximization formula. 0 Taking the absolute value of each element of {|a1|,|a2|,…,|a n ∣}, for {∣a1∣,∣a2∣,…,∣a n || Arrange in descending order to obtain vector B = {b1, b2, ..., b n}, the quantized vector T' corresponding to vector B = (t1', t2', ..., t i ',…t n '), where t i ' indicates the original i The number of non-zeros in T' is M (M = 1, ..., N), then when j∈(M+1, ..., N), t i' are all zero. By solving equation (2), we can get an M that maximizes the result of (2), that is, the target value. After obtaining M, we get the number of elements in T with an absolute value of 1, and according to the corresponding relationship between the descending vector and the normalized vector elements, we determine which elements in T have an absolute value of 1, and then according to the positive and negative of the elements in the normalized vector, we determine the positive and negative of the elements in T with an absolute value of 1, that is, if the element in the normalized vector is positive, the corresponding element in T is +1, and if the element in the normalized vector is negative, the corresponding element in T is -1.

[0121] In this embodiment, in the process of calculating the approximate integer vector of the vector to be transferred based on the cosine similarity formula, the nonlinear integer problem is transformed into a linear optimization problem, that is, it is transformed into a problem of solving the maximization formula, and the amount of calculation for the optimization solution is reduced to O(n*log n), which is smaller than the amount of calculation O(N*log n) using the function transformation method. 2 ), this embodiment greatly reduces the amount of computation in the process of federated learning information transmission, thereby improving the communication efficiency while also improving the overall efficiency of the federated learning modeling process.

[0122] Furthermore, after step S201, the method further includes:

[0123] Step S202, projecting the vector to be transferred onto the approximate integer vector to obtain a scaling factor of the approximate integer vector relative to the vector to be transferred;

[0124] Step S203: Send the scaling factor to the receiver, so that after receiving the approximate integer vector, the receiver scales the approximate integer vector based on the scaling factor and continues to perform the federated learning task based on the scaling result.

[0125] Based on the quantization of the vector to be transferred based on the cosine similarity formula, the sender can calculate the scaling factor of the approximate integer vector relative to the vector to be transferred, and send the scaling factor to the receiver, so that the receiver can use the scaling factor to scale the approximate integer vector after receiving the approximate integer vector, so that the difference between the scaled approximate integer vector and the vector to be transferred is smaller, thereby minimizing the information loss caused by the quantization operation, thereby achieving privacy protection effects and improving communication efficiency while further ensuring that the model can converge and ensuring the federated learning modeling effect.

[0126] Specifically, the sender may project the vector to be transmitted onto the approximate integer vector to obtain a scaling factor of the approximate integer vector relative to the vector to be transmitted. The vector to be transmitted may be projected onto the approximate integer vector in an orthogonal projection manner to obtain a scaling factor.

[0127] To achieve the scaling factor based on projection, we can use Figure 5 The equation (3) shown represents the scaling factor, and the scaling factor is obtained by equation (3), where M is the maximized M.

[0128] Furthermore, in order to further reduce the difference between the scaled approximate integer vector and the vector to be transferred, the scaling factor may also include a positive scaling factor and a negative scaling factor. Step S202 includes:

[0129] Step S2021, extracting a positive element vector and a negative element vector from the vector to be transferred, and extracting a positive integer vector and a negative integer vector from the approximate integer vector;

[0130] The sender can extract positive element vectors and negative element vectors from the vector to be transmitted, and extract positive integer vectors and negative integer vectors from the approximate integer vector. Extracting positive element vectors means keeping the positive elements in the vector to be transmitted unchanged, and converting all other elements to 0. The extraction process of negative element vectors, positive integer vectors, and negative integer vectors is the same.

[0131] Step S2022, projecting the positive element vector onto the positive integer vector to obtain the positive scaling factor, and projecting the negative element vector onto the negative integer vector to obtain the negative scaling factor;

[0132] The sender projects the positive element vector onto a positive integer vector to obtain a positive scaling factor, and projects the negative element vector onto a negative integer vector to obtain a negative scaling factor. The projection method may be an orthogonal projection method. Based on the above steps and principles, the above formula (3) may be used to calculate the positive scaling factor and the negative scaling factor; w in formula (3) i is a positive element in W, t i When is a positive element in T, a positive scaling factor is calculated; w in formula (3) i is a negative element in W, t i When is a negative element in T, a negative scaling factor is calculated.

[0133] The step S203 includes:

[0134] Step S2031, sending the positive scaling coefficient and the negative scaling coefficient to the receiver, so that after receiving the approximate integer vector, the receiver scales the positive elements in the approximate integer vector based on the positive scaling coefficient, and scales the negative elements in the approximate integer vector based on the negative scaling coefficient, so as to scale the approximate integer vector and continue to perform the federated learning task based on the scaling result.

[0135] The sender sends the positive scaling factor and the negative scaling factor to the receiver. After receiving the approximate integer vector sent by the sender, the receiver can scale the positive elements in the approximate integer vector based on the positive scaling factor, that is, multiply all the positive elements in the approximate integer vector by the positive scaling factor; and then scale the negative elements in the approximate integer vector based on the negative scaling factor, that is, multiply all the negative elements in the approximate integer vector by the negative scaling factor; by scaling using the positive scaling factor and the negative scaling factor, the scaling of the approximate integer vector is completed, and the receiver continues to perform subsequent federated learning tasks based on the scaling result, that is, based on the scaled approximate integer vector.

[0136] The positive scaling factor and the negative scaling factor of the approximate integer vector relative to the vector to be transferred are obtained by calculation, and the approximate integer vector is scaled by the positive scaling factor and the negative scaling factor, so that the difference in length between the scaled approximate integer vector and the vector to be transferred is smaller.

[0137] Furthermore, the step S30 includes:

[0138] Step S301, Huffman encodes the approximate quantization information, and sends the encoded approximate quantization information to a receiver, so that the receiver decodes the encoded approximate quantization information and continues to perform the federated learning task based on the decoding result.

[0139] In order to further compress the approximate quantization information and reduce communication overhead, the sender can perform Huffman coding on the approximate vector information and send the coded approximate quantization information to the receiver. After receiving the coded approximate quantization information, the receiver can decode the coded approximate quantization information and continue to perform the federated learning task based on the decoding result. Among them, Huffman coding is an entropy coding (weight coding) algorithm for lossless data compression.

[0140] Further, based on the above first or second embodiment, a third embodiment of the information transmission method of the present invention is proposed. In this embodiment, step S30 includes:

[0141] Step S302, transforming the approximate quantization information according to a preset transformation rule to obtain transformation information;

[0142] The sender may transform the approximate quantization information according to a preset transformation rule to obtain the transformed approximate quantization information (referred to as transformed information). The preset transformation rule may be a pre-set transformation rule. After the approximate quantization information is transformed according to the transformation rule, all or part of the information in the approximate quantization information is changed. Since the transformation is based on the transformation rule, the change is based on the law possessed by the transformation rule. For example, the transformation rule may be to change all positive numbers in the approximate quantization information into negative numbers, and all negative numbers into positive numbers. It should be noted that multiple sets of transformation rules may be preset, and the sender may select different transformation rules to transform the approximate quantization information each time the information is transmitted. For example, the participating devices may use different transformation rules for transformation in each round of sending local gradient information. Since the original approximate quantized information is transformed each time the information is sent, it is difficult for a third-party attacker to obtain the original approximate quantized information based on the transformed information without a transformation rule, thereby achieving privacy protection; further, if the transformation rule used each time the information is transmitted is random, or even different each time, it will be even more difficult for an attacker to obtain the original approximate quantized information based on the transformed information each time, thereby further improving the privacy protection effect.

[0143] Step S303: Send the transformation information and the transformation rule to a receiver, so that the receiver can restore the transformation information based on the transformation rule to obtain the approximate quantization information, and continue to perform the federated learning task based on the approximate quantization information.

[0144] After obtaining the transformation information, the sender can send the transformation information and the transformation rules to the receiver. After receiving the transformation information and the transformation rules, the receiver restores the transformation information to obtain the information to be transmitted, that is, based on the transformation information, performs a transformation operation opposite to the transformation rule to restore the transformation information to obtain approximate quantitative information. After obtaining the approximate quantitative information, the receiver can continue to perform subsequent federated learning tasks. For example, when the receiver is a coordinating device, and the restored approximate quantitative information is the quantitative information corresponding to the local model parameter information sent by the participating devices, the receiver can fuse the various approximate quantitative information to obtain the global model parameter information, and then use the global model parameter information as the information to be transmitted. It should be noted that the transformation rule sent by the sender to the receiver can be a parsable description file, and the receiver can parse the description file to obtain the transformation rule and execute the transformation rule.

[0145] In this embodiment, by obtaining the information to be transmitted in the federated learning task, the approximate quantized information of the information to be transmitted is calculated, and the approximate quantized information is transformed according to the preset transformation rules to obtain the transformed information, and the transformed information and the change rules are sent to the receiver, so that the receiver can restore the transformed information according to the transformation rules to obtain the approximate quantized information, and continue to perform the federated learning task based on the approximate quantized information. In this embodiment, since the information transmitted is the approximate quantized information after the transformation according to the transformation rules, not the information to be transmitted, nor the approximate quantized information, the third attacker can only obtain the transformed approximate quantized information. In the absence of the transformation rules, the information to be transmitted itself cannot be restored, and the private information in the federated learning cannot be stolen, which improves the privacy protection effect of the federated learning. Compared with the method of using encryption to protect, in this embodiment, since the pre-set transformation rules are used to transform the approximate quantized information, the transformation rules can be set to be simpler, and there is no need to use complex encryption algorithms and encryption keys for calculation, thereby reducing the amount of calculation in the information transmission process, improving the information transmission efficiency in the federated learning process, and thus improving the efficiency of federated learning modeling. It should be noted that in this embodiment, the sender transmits the transformation rule itself to the receiver, rather than the encryption algorithm key. Therefore, the transformation rule in this embodiment is different from the encryption algorithm. Therefore, the scheme of using the transformation rule for transformation processing in this embodiment is different from the scheme of using the encryption algorithm for privacy protection. Moreover, the transformation rule can be more random than the encryption algorithm, that is, different customized transformation rules can be used for transformation processing in each transmission process, so that third-party attackers have no rules to follow and cannot use existing encryption algorithms to crack.

[0146] Furthermore, when the approximate quantization information is an approximate integer vector, the sender transforms the approximate integer vector according to a preset transformation rule to obtain a transformation vector, and uses the transformation vector as the transformation information. Among them, some elements in the transformation vector are different from the elements at the corresponding positions in the approximate integer vector. That is, after the transformation is processed according to the transformation rule, a transformation vector is obtained, and the length of the transformation vector can be the same as that of the original approximate integer vector, then some elements or all elements in the transformation vector are different from the elements at the corresponding positions in the approximate integer vector. That is, the transformation rule can be a set transformation rule, according to which some elements in the approximate integer vector can be changed, then the changed elements can be restored according to the transformation rule, so that the receiver can restore the approximate integer vector without causing information loss.

[0147] In this embodiment, the approximate integer vector corresponding to the vector to be transferred is transformed according to a preset transformation rule to obtain a transformation vector, and the transformation vector and the transformation rule are sent to the recipient, so that the recipient uses the transformation rule to restore the transformation vector to obtain the approximate integer vector. In this way, even if a third-party attacker can restore the approximate integer vector to obtain the vector to be transferred, because what is transferred in this embodiment is the transformation vector instead of the approximate integer vector, the third-party attacker has an additional layer of cracking difficulty, thereby further improving the privacy protection effect in the process of federated learning information transmission.

[0148] In order to further increase the difficulty of attack for attackers and improve the privacy protection effect, in this embodiment, the sender can transform the approximate integer vector according to a preset transformation rule to obtain an orthogonal vector of the approximate integer vector, and use the orthogonal vector as the transformation vector.

[0149] Among them, an orthogonal vector refers to a vector whose inner product with the approximate integer vector is zero. That is, the approximate integer vector is transformed by a transformation rule so that the inner product of the transformed vector and the vector before the transformation is zero. There are multiple transformation rules that make the inner product of the transformed vector and the vector before the transformation zero, and multiple sets of different transformation rules can be set in advance. For example, the transformation rule can be set as follows: when the elements in the approximate integer vector obtained by quantization are 0, -1 or +1, and the number of elements of -1 is equal to the number of elements of +1, all -1 in the approximate integer vector can be changed to +1, and the remaining elements remain unchanged, thereby obtaining an orthogonal vector of the approximate integer vector.

[0150] By using transformation rules to transform the approximate integer vector to obtain the orthogonal vector of the approximate integer vector, the orthogonal vector is sent to the receiver as the transformation vector, so that the third-party attacker can only obtain the orthogonal vector. The direction of the orthogonal vector is perpendicular to the approximate integer vector, so the third-party attacker can hardly deduce the input data of the model based on the orthogonal vector, and thus cannot obtain the private data. In addition, there are many transformation rules that can obtain the orthogonal vector, and the third-party attacker cannot know which one it is, making it more difficult to crack and obtain the private information, thereby further improving the privacy protection effect in the process of federated learning information transmission.

[0151] Furthermore, in order to avoid the third-party attacker from obtaining the transformation information and the transformation rules at the same time, the sender can send the transformation information and the transformation rules to the receiver based on different communication channels. Specifically, two different communication channels can be established between the sender and the receiver, and both of these communication channels can be constructed using existing communication methods, which are not limited here. By sending the transformation rules and transformation information separately through different communication channels, it is difficult for a third-party attacker to intercept both types of information at the same time, and thus it is difficult to steal the private information in federated learning, which further increases the privacy protection of federated learning.

[0152] Alternatively, the sender can encrypt the transformation rule, and send the transformation information and the encrypted transformation rule to the receiver, and the same communication channel can be used for transmission. Encryption can be performed using a commonly used encryption algorithm, such as RSA (an asymmetric encryption algorithm), EIGmal (a dual-key cryptographic system based on discrete logarithm problem) and DSA (Digital Signature Algorithm). The receiver can use a decryption algorithm corresponding to the encryption algorithm of the sender to decrypt, and after decrypting to obtain the transformation rule, the transformation information is restored using the transformation rule. Through encryption, even if a third-party attacker obtains the transformation rule, it is the encrypted transformation rule and cannot be used to restore the transformation information, thereby increasing the privacy protection of federated learning. Moreover, compared with the existing privacy protection method that uses an encryption algorithm to directly encrypt the information to be transmitted, in this embodiment, only the transformation rule needs to be encrypted, and the data volume of the transformation rule is much smaller than the information to be transmitted, so that the amount of calculation in the information transmission process will not be too large, thereby speeding up the information transmission efficiency in the federated learning process.

[0153] Alternatively, the transformation rule may be encrypted, and the transformation information and the encrypted transformation rule may be sent to the receiver based on different communication channels, thereby achieving a further privacy protection effect.

[0154] In addition, the sender may encrypt the scaling factor and send it to the receiver, or send the scaling factor and the transformation information to the receiver based on different communication channels.

[0155] In addition, the embodiment of the present invention also proposes an information transmission device, which is deployed on a device participating in federated learning, referring to Figure 6 , the information transmission device comprises:

[0156] An acquisition module 10, used to acquire information to be transmitted in a federated learning task;

[0157] A processing module 20, configured to perform approximate quantization processing on the information to be transmitted to obtain approximate quantization information of the information to be transmitted;

[0158] The sending module 30 is used to send the approximate quantization information to a receiving party, so that the receiving party can continue to perform the federated learning task based on the approximate quantization information.

[0159] Furthermore, the information to be transmitted is a vector to be transmitted, and the processing module 20 is further used to: perform approximate quantization processing on the vector to be transmitted according to a cosine similarity formula to obtain an approximate integer vector of the vector to be transmitted, and use the approximate integer vector as the approximate quantized information.

[0160] Furthermore, the processing module 20 includes:

[0161] An acquiring unit, used for acquiring a descending vector corresponding to the vector to be transferred;

[0162] a calculation unit, configured to substitute the descending vector and the candidate quantity value into a maximization formula for maximizing a target value for calculation, and select the candidate quantity value that maximizes the target value as the target quantity value, wherein the maximization formula is constructed based on a cosine similarity formula, and in the maximization formula, the target value is equal to a result obtained by summing the target elements in the descending vector and dividing by the target quantity under the square root, and the target element is an element in the descending vector whose sequence number is less than or equal to the candidate quantity value;

[0163] The quantization unit is used to perform a quantization value allocation operation based on the target quantity value to obtain an approximate integer vector corresponding to the vector to be transferred.

[0164] Furthermore, the acquisition unit includes:

[0165] A processing subunit, used for performing unitization processing on the vector to be transferred to obtain a unitized vector;

[0166] A descending subunit, used for taking the absolute value of each element in the normalized vector and arranging them in descending order to obtain a descending vector;

[0167] The quantization unit comprises:

[0168] A determination subunit, configured to determine elements in the descending vector whose sequence numbers are less than or equal to the target quantity value as descending elements, and to use elements in the normalized elements corresponding to the descending elements as retained elements;

[0169] The quantization subunit is used to assign a quantization value to each element in the normalized vector to obtain an approximate integer vector corresponding to the vector to be transmitted, wherein positive 1 is assigned to the elements greater than zero in the retained elements, negative 1 is assigned to the elements less than zero in the retained elements, and 0 is assigned to the elements other than the retained elements in the normalized vector.

[0170] Furthermore, the processing module 20 includes:

[0171] A substitution unit, used for substituting a preset integer vector and the vector to be transferred into a cosine similarity formula, wherein the value of each element of the preset integer vector is to be determined;

[0172] A solving unit is used to solve the cosine similarity formula to obtain the value of each element of the preset integer vector, and use the preset integer vector with the determined value of each element as the approximate integer vector of the vector to be transferred, wherein the preset integer vector with the determined value of each element maximizes the cosine value of the cosine similarity formula.

[0173] Furthermore, the information transmission device also includes:

[0174] A projection module, used for projecting the vector to be transferred onto the approximate integer vector to obtain a scaling factor of the approximate integer vector relative to the vector to be transferred;

[0175] The sending module 30 is further used to send the scaling factor to the receiving party, so that after receiving the approximate integer vector, the receiving party scales the approximate integer vector based on the scaling factor and continues to perform the federated learning task based on the scaling result.

[0176] Further, the scaling factor includes a positive scaling factor and a negative scaling factor, and the projection module includes:

[0177] An extraction unit, used to extract a positive element vector and a negative element vector from the vector to be transferred, and to extract a positive integer vector and a negative integer vector from the approximate integer vector;

[0178] a projection unit, configured to project the positive element vector onto the positive integer vector to obtain the positive scaling factor, and project the negative element vector onto the negative integer vector to obtain the negative scaling factor;

[0179] The sending module 30 includes:

[0180] The first sending unit is used to send the positive scaling coefficient and the negative scaling coefficient to the receiving party, so that after receiving the approximate integer vector, the receiving party scales the positive elements in the approximate integer vector based on the positive scaling coefficient and scales the negative elements in the approximate integer vector based on the negative scaling coefficient, so as to scale the approximate integer vector and continue to perform the federated learning task based on the scaling result.

[0181] Furthermore, the sending module 30 includes:

[0182] The encoding unit is used to perform Huffman encoding on the approximate quantization information, and send the encoded approximate quantization information to a receiving party, so that the receiving party can decode the encoded approximate quantization information and continue to perform the federated learning task based on the decoding result.

[0183] Furthermore, the sending module 30 includes:

[0184] A transformation unit, configured to transform the approximate quantization information according to a preset transformation rule to obtain transformation information;

[0185] The second sending unit is used to send the transformation information and the transformation rule to a receiving party, so that the receiving party can restore the transformation information based on the transformation rule to obtain the approximate quantization information, and continue to perform the federated learning task based on the approximate quantization information.

[0186] The expanded contents of the specific implementation manner of the information transmission device of the present invention are basically the same as the embodiments of the above-mentioned information transmission method, and will not be elaborated here.

[0187] In addition, an embodiment of the present invention further proposes a computer-readable storage medium, on which an information transmission program is stored. When the information transmission program is executed by a processor, the steps of the information transmission method described below are implemented.

[0188] The various embodiments of the information transmission device and computer-readable storage medium of the present invention may refer to the various embodiments of the information transmission method of the present invention, and will not be described in detail here.

[0189] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0190] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0191] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0192] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An information transmission method, characterized in that: The information transmission method is applied to a device participating in federated learning, and the information transmission method comprises the following steps: Get the information to be transmitted in the federated learning task; Performing approximate quantization processing on the information to be transmitted to obtain approximate quantization information of the information to be transmitted; Sending the approximate quantization information to a recipient, so that the recipient continues to perform the federated learning task based on the approximate quantization information; The information to be transmitted is a vector to be transmitted, and the step of performing approximate quantization processing on the information to be transmitted to obtain approximate quantized information of the information to be transmitted includes: Performing approximate quantization processing on the vector to be transferred according to the cosine similarity formula to obtain an approximate integer vector of the vector to be transferred, and using the approximate integer vector as the approximate quantization information; The step of performing approximate quantization processing on the vector to be transferred according to the cosine similarity formula to obtain an approximate integer vector of the vector to be transferred comprises: Obtaining a descending vector corresponding to the vector to be transferred; Substituting the descending vector and the candidate quantity value into a maximization formula for maximizing the target value for calculation, selecting the candidate quantity value that maximizes the target value as the target quantity value, wherein the maximization formula is constructed based on the cosine similarity formula, in which the target value is equal to the result obtained by summing the target elements in the descending vector and dividing by the target quantity under the square root, and the target element is an element in the descending vector whose sequence number is less than or equal to the candidate quantity value; Performing a quantization value allocation operation based on the target quantity value to obtain an approximate integer vector corresponding to the vector to be transferred; Or the step of performing approximate quantization processing on the vector to be transferred according to the cosine similarity formula to obtain an approximate integer vector of the vector to be transferred includes: Substituting a preset integer vector and the vector to be transferred into a cosine similarity formula, wherein the value of each element of the preset integer vector is to be determined; The cosine similarity formula is solved to obtain the values ​​of each element of the preset integer vector, and the preset integer vector with the determined values ​​of each element is used as the approximate integer vector of the vector to be transmitted, wherein the preset integer vector with the determined values ​​of each element maximizes the cosine value of the cosine similarity formula.

2. The information transmission method according to claim 1, characterized in that: The step of obtaining the descending vector corresponding to the vector to be transferred comprises: Performing unitization processing on the vector to be transferred to obtain a unitized vector; Taking the absolute value of each element in the normalized vector and arranging them in descending order to obtain a descending vector; The step of performing a quantization value allocation operation based on the target quantity value to obtain an approximate integer vector corresponding to the vector to be transferred includes: Determine the elements in the descending vector whose sequence numbers are less than or equal to the target quantity value as descending elements, and use the elements in the normalized elements corresponding to the descending elements as retained elements; Assign a quantized value to each element in the normalized vector to obtain an approximate integer vector corresponding to the vector to be transmitted, wherein positive 1 is assigned to the elements greater than zero in the retained elements, negative 1 is assigned to the elements less than zero in the retained elements, and 0 is assigned to the elements other than the retained elements in the normalized vector.

3. The information transmission method according to claim 1, characterized in that: After the step of performing approximate quantization processing on the vector to be transferred according to the cosine similarity formula to obtain an approximate integer vector of the vector to be transferred and using the approximate integer vector as the approximate quantization information, the method further includes: Projecting the vector to be transferred onto the approximate integer vector to obtain a scaling factor of the approximate integer vector relative to the vector to be transferred; The scaling factor is sent to the receiver, so that after receiving the approximate integer vector, the receiver scales the approximate integer vector based on the scaling factor and continues to perform the federated learning task based on the scaling result.

4. The information transmission method according to claim 3, characterized in that: The scaling factor includes a positive scaling factor and a negative scaling factor. The step of projecting the vector to be transferred onto the approximate integer vector to obtain the scaling factor of the approximate integer vector relative to the vector to be transferred includes: Extracting a positive element vector and a negative element vector from the vector to be transferred, and extracting a positive integer vector and a negative integer vector from the approximate integer vector; Projecting the positive element vector onto the positive integer vector to obtain the positive scaling factor, and projecting the negative element vector onto the negative integer vector to obtain the negative scaling factor; The step of sending the scaling factor to the receiving party so that the receiving party scales the approximate integer vector based on the scaling factor after receiving the approximate integer vector, so as to continue to perform the federated learning task based on the scaling result includes: The positive scaling factor and the negative scaling factor are sent to the receiver, so that after receiving the approximate integer vector, the receiver scales the positive elements in the approximate integer vector based on the positive scaling factor and scales the negative elements in the approximate integer vector based on the negative scaling factor, so as to scale the approximate integer vector and continue to perform the federated learning task based on the scaling result.

5. The information transmission method according to any one of claims 1 to 4, characterized in that: The step of sending the approximate quantization information to a receiver so that the receiver continues to perform the federated learning task based on the approximate quantization information includes: The approximate quantization information is Huffman encoded, and the encoded approximate quantization information is sent to a receiver, so that the receiver can decode the encoded approximate quantization information and continue to perform the federated learning task based on the decoding result.

6. The information transmission method according to any one of claims 1 to 4, characterized in that: The step of sending the approximate quantization information to a receiver so that the receiver continues to perform the federated learning task based on the approximate quantization information includes: Transforming the approximate quantization information according to a preset transformation rule to obtain transformation information; The transformation information and the transformation rule are sent to a receiver, so that the receiver can restore the transformation information based on the transformation rule to obtain the approximate quantization information, and continue to perform the federated learning task based on the approximate quantization information.

7. An information transmission device, characterized in that: The information transmission device is deployed on a device participating in federated learning, and the information transmission device includes: The acquisition module is used to obtain the information to be transmitted in the federated learning task; a processing module, configured to perform approximate quantization processing on the information to be transmitted to obtain approximate quantization information of the information to be transmitted; the information to be transmitted is a vector to be transmitted, and the processing module is further configured to perform approximate quantization processing on the vector to be transmitted according to a cosine similarity formula to obtain an approximate integer vector of the vector to be transmitted, and use the approximate integer vector as the approximate quantization information; A sending module, configured to send the approximate quantization information to a receiving party, so that the receiving party can continue to perform the federated learning task based on the approximate quantization information; The processing module comprises: An acquiring unit, used for acquiring a descending vector corresponding to the vector to be transferred; a calculation unit, configured to substitute the descending vector and the candidate quantity value into a maximization formula for maximizing a target value for calculation, and select the candidate quantity value that maximizes the target value as the target quantity value, wherein the maximization formula is constructed based on a cosine similarity formula, and in the maximization formula, the target value is equal to a result obtained by summing the target elements in the descending vector and dividing by the target quantity under the square root, and the target element is an element in the descending vector whose sequence number is less than or equal to the candidate quantity value; A quantization unit, configured to perform a quantization value allocation operation based on the target quantity value to obtain an approximate integer vector corresponding to the vector to be transferred; Or the processing module includes: A substitution unit, used for substituting a preset integer vector and the vector to be transferred into a cosine similarity formula, wherein the value of each element of the preset integer vector is to be determined; A solving unit is used to solve the cosine similarity formula to obtain the value of each element of the preset integer vector, and use the preset integer vector with the determined value of each element as the approximate integer vector of the vector to be transferred, wherein the preset integer vector with the determined value of each element maximizes the cosine value of the cosine similarity formula.

8. An information transmission device, characterized in that: The information transfer device comprises: a memory, a processor, and an information transfer program stored in the memory and executable on the processor, wherein the information transfer program implements the steps of the information transfer method according to any one of claims 1 to 6 when executed by the processor.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an information transmission program, and when the information transmission program is executed by the processor, the steps of the information transmission method according to any one of claims 1 to 6 are implemented.

Images