Aerial federated learning method based on rare 1-bit quantization, and a server and device therefor.

Abstract

This invention provides a new aerial federation learning method based on rare 1-bit quantization that enables compression and transmission, as well as a server and device for that purpose. [Solution] The airborne federation learning method based on rare 1-bit quantization for a computer-equipped device of the present invention comprises the steps of: receiving a global parametric vector from a server in each communication round; deriving a local gradient vector by applying a first-order approximation algorithm using a local data set based on the global parametric vector; compressing the local gradient vector by 1-bit quantization for each layer and then sparse it; and transmitting the sparsed 1-bit quantized local gradient vector to the server via an uplink channel in an analog manner. The server aggregates and restores the sparsed 1-bit quantized local gradient vector, updates the global parametric vector, and then broadcasts it.

Description

[Technical Field]

[0001] This invention relates to a Federated Learning Over-the-Air method based on Sparse One-Bit Quantization, as well as a server and device therefor. [Background technology]

[0002] Machine learning (ML), which has become widespread in recent years, provides a variety of application services through big data. However, centralized data center learning faces personal data protection issues because parameter servers (PS) collect data generated by each device.

[0003] Federated Learning (FL) is an ML paradigm that emerged to solve these problems. It aims to train a single global learning model using data stored on numerous local devices, based on the adjustment of the Processing System (PS). FL repeatedly performs a communication round, aggregating the local gradients of the trained local learning models until the global learning model converges. Therefore, unlike centralized data center learning, FL does not collect raw data, providing an opportunity to utilize the computing resources of each device while protecting personal information.

[0004] FL allows for the selection of various learning models depending on the application service being provided. However, FL may experience excessive communication costs that can become bottlenecks due to the numerous parameters that make up the learning model. For example, when providing an image classification service, it is necessary to select artificial neural networks such as ResNet-152 with 6.3 million parameters or VGG-16 with 138 million parameters. Therefore, FL considers gradient compression methods to reduce the communication cost per device. Quantization is a system that restricts each parameter to be represented by a finite number of bits in the local gradient. Scarcity is a system that removes some parameters in the local gradient, retaining only the important parameters.

[0005] Hybrid systems combine quantization and scarcity. Information loss due to local gradient compression leads to errors in global gradient estimation, resulting in reduced convergence and learning performance. Therefore, gradient compression systems tend to incorporate error feedback mechanisms to compensate for such information loss in subsequent communication rounds.

[0006] Despite gradient compression systems, FL still faces a bottleneck because communication costs are affected by the number of devices participating in each communication round. Therefore, FL has recently evolved into FL Over-the-Air (FLOA) by integrating with Over-the-Air Computation (OAC), which leverages the waveform superposition characteristics of Multiple Access Channels (MAC) to transform the air into a computer performing computing and communication functions.

[0007] In FLOA, all devices simultaneously transmit local gradients over the same time-frequency resources, and the PS receives the result of the analog aggregation of these local gradients in the air. Therefore, FLOA can integrate communication and computing to achieve higher bandwidth efficiency than existing FL. FLOA can solve the problem that the communication cost linearly increases with the number of devices participating in each communication round regarding high communication costs. However, the problem that the communication cost increases due to the number of intermediate variables in the learning model has not yet been solved. Also, FLOA faces the typical problem of OAC that the signals transmitted from each device are distorted by noise and fading.

[0008] Therefore, many prior studies have focused on gradient compression systems and power control techniques to reduce communication costs and signal distortion. Gradient compression systems and power control techniques that can control compression and aggregation errors have a great impact on the performance of the learning model. However, existing prior studies have the problem that they cannot optimally control such compression and aggregation errors because they adopt a fixed compression rate regardless of the channel state.

Summary of the Invention

Problems to be Solved by the Invention

[0009] The present invention has been made in view of the above conventional problems, and an object of the present invention is to provide a method for over-the-air federated learning based on sparse one-bit quantization enabling new compression and transmission, as well as a server and a device therefor.

Means for Solving the Problems

[0010] According to one aspect of the present invention, a method for over-the-air federated learning based on sparse one-bit quantization is provided.

[0011] An airborne federation learning method based on rare 1-bit quantization for a computer device according to one aspect of the present invention, made to achieve the above objective, comprises the steps of: receiving a global parametric vector from a server in each communication round; deriving a local gradient vector by applying a first-order approximation algorithm using a local data set based on the global parametric vector; compressing the local gradient vector by 1-bit quantization for each layer and then sparse it; and transmitting the sparsed 1-bit quantized local gradient vector to the server via an uplink channel in an analog manner, wherein the server aggregates and restores the sparsed 1-bit quantized local gradient vector, updates the global parametric vector, and then broadcasts it.

[0012] The step of compressing each layer by 1-bit quantization and then scarcely applying it may include the steps of calculating the size-scaling elements of the local gradient vector for each layer, and deriving the scarce 1-bit quantized local gradient vector using the calculated size-scaling elements and the scarcity masking indicator for each layer. The aforementioned layer-specific scarcity masking indicator is an indicator that determines whether a layer is transmitted and can be transmitted from the server in each communication round. Layers for which the scarcity masking indicator is 0 may be excluded from transmission. The aforementioned layer may be a layer that constitutes a deep learning model. Deriving the aforementioned local gradient vector may involve calculating a local gradient vector corrected using the previous round compression error vector. Updating the global parametric vector may include correcting the analog signal received via the uplink with an amplitude scaling element and transmission power to reconstruct the gradient vector, aggregating the reconstructed gradient vector, calculating and aggregating the global gradient vector, and updating the global parametric vector using the aggregated global gradient vector. The server may further include transmitting the reconstructed gradient vector to the device.

[0013] To achieve the above objective, an aerial federation learning method based on rare 1-bit quantization for a server equipped with a computer according to one aspect of the present invention comprises: (a) defining an amplitude scaling element and a binary rare masking indicator based on a loss function and constraints in each communication round and transmitting them to each device; (b) receiving a signal from each device which is a rared 1-bit quantized local gradient vector; (c) correcting the received signal with an amplitude scaling element and transmission power to reconstruct the gradient vector; (d) aggregating the reconstructed gradient vectors and calculating and aggregating a global gradient vector; and (e) updating a global parametric vector using the aggregated global gradient vector.

[0014] The (a) step involves determining the amplitude scaling element and the binary scarcity masking indicator in parallel, and the binary scarcity masking indicator may be determined after the amplitude scaling element has been determined. The aforementioned scarcity masking indicator may be an indicator that determines whether or not the layer is being transmitted.

[0015] According to one aspect of the present invention, a device is provided for performing a Federated Learning Over-the-Air method based on Sparse One-Bit Quantization.

[0016] A device according to one aspect of the present invention made to achieve the above objective comprises a communication unit, a memory for storing at least one instruction word, and a processor for executing the instruction word stored in the memory. The processor is configured to receive a global parametric vector from a server in each communication round when the instruction word is executed, derive a local gradient vector by applying a first-order approximation algorithm using a local data set based on the global parametric vector, compress the local gradient vector by 1-bit quantization for each layer and then sparse it, and transmit the sparsed 1-bit quantized local gradient vector to the server via an uplink channel in an analog manner. The server aggregates and restores the sparsed 1-bit quantized local gradient vector, updates the global parametric vector, and then broadcasts it.

[0017] A server according to one aspect of the present invention made to achieve the above objective comprises a communication unit, a memory for storing at least one instruction word, and a processor for executing the instruction word stored in the memory, wherein when the instruction word is executed, the processor is configured to (a) determine an amplitude scaling element and a binary scarcity masking indicator based on a loss function and constraints in each communication round and transmit them to each device, (b) receive a signal from each device which is a scarce 1-bit quantized local gradient vector, (c) correct the received signal with an amplitude scaling element and transmission power to reconstruct the gradient vector, (d) aggregate the reconstructed gradient vectors to calculate and aggregate a global gradient vector, and (e) update a global parametric vector using the aggregated global gradient vector. [Effects of the Invention]

[0018] According to the airborne federated learning method based on rare 1-bit quantization of the present invention, and the server and device therefor, it is possible to perform new compression and transmission that jointly optimize power and error control based on dynamic control of the compression ratio based on scarcity, thereby reducing communication costs. [Brief explanation of the drawing]

[0019] [Figure 1] This diagram schematically shows a system configuration for performing an aerial federated learning method based on rare 1-bit quantization according to one embodiment of the present invention. [Figure 2] This figure shows pseudocode for a feasible solution for a disassembly substrate according to one embodiment of the present invention. [Figure 3] This is a flowchart showing an aerial federated learning method based on rare 1-bit quantization performed on each device according to one embodiment of the present invention. [Figure 4] This is a flowchart showing an aerial federated learning method based on rare 1-bit quantization performed on each server according to one embodiment of the present invention. [Figure 5] This figure shows the results of a preliminary experiment based on the dataset. [Figure 6] This figure shows the setting of experimental parameters according to one embodiment of the present invention. [Figure 7] This figure shows the evaluation of learning performance in error-free channels using both a conventional method and one embodiment of the present invention. [Figure 8] This figure shows the evaluation of learning performance in a fading channel using a conventional method and an embodiment of the present invention. [Figure 9] This diagram compares the communication efficiency of a conventional method and an embodiment of the present invention. [Figure 10] This figure compares the effects of a conventional device group and a device group according to the embodiment of the present invention. [Figure 11] This diagram compares the impact of network conditions under conventional and embodiment 1. [Figure 12] This diagram schematically shows the internal configuration of a device / server according to one embodiment of the present invention. [Modes for carrying out the invention]

[0020] As used herein, singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “composed of” or “including” should not be interpreted as necessarily including all of the many components or stages described in the specification, and may not include some of them, or may include further components or stages. Furthermore, terms such as “…part” or “module” as used in this specification mean a unit that processes at least one function or operation, which may be embodied in hardware or software, or in a combination of hardware and software.

[0021] Hereinafter, specific examples of embodiments for carrying out the present invention will be described in detail with reference to the drawings.

[0022] Figure 1 is a schematic diagram showing a system configuration for performing an airborne federated learning method based on rare 1-bit quantization according to one embodiment of the present invention, and Figure 2 is a diagram showing pseudocode for a feasible solution of the decomposition base according to one embodiment of the present invention.

[0023] As shown in Figure 1, the system according to one embodiment of the present invention is a federated learning system based on a wireless network consisting of a single server 110 and K devices 120.

[0024] For the sake of understanding and explanation, we will first explain the systems learning model.

[0025] ≪System Learning Model≫

[0026] In this system, device k is D k Assuming we have a local data set consisting of data samples, we have a local loss function with a parameter vector w. Define TIFF2026093311000002.tif9143 as shown in Formula 1.

[0027]

number

[0028] Here, TIFF2026093311000004.tif10129 is a feature vector x k,d and label y k,d The loss function of sample d, which is composed of the above, is shown. The local loss function of all devices is also shown. Based on TIFF2026093311000005.tif10129, global loss function Define TIFF2026093311000006.tif9129 as shown in formula 2.

[0029]

number

[0030] Here, D Defined as TIFF2026093311000008.tif15148.

[0031] Therefore, associative learning uses a global loss function as shown in Equation 3. Minimize TIFF2026093311000009.tif9129 to find the optimal parametric vector w * The goal is to define [this].

[0032]

number

[0033] The FL system configures the training process between the server 110 and the devices in a distributed manner, and raw samples are not transmitted directly from each device to the server 110. In other words, the server 110 does not collect raw samples from each device.

[0034] In each communication round TIFF2026093311000011.tif9148 Server 110 broadcasts the mediating variable vector w to all devices. t to broadcast.

[0035] Each device uses a first-order approximation algorithm to update the received mediating variable w based on the local data set in order to minimize the local loss function and derives the local gradient vector as shown in Equation 4. t to update and derive the local gradient vector as in Equation 4.

[0036]

Number

[0037] Here, TIFF2026093311000013.tif10148 indicates the gradient with respect to w. t with respect to TIFF2026093311000014.tif7129.

[0038] Each device transmits the derived local gradient vector TIFF2026093311000015.tif9128 to Server 110, and Server 110 aggregates the local gradient vectors TIFF2026093311000016.tif9128 transmitted from each device to derive the global gradient vector g t as shown in Equation 5.

[0039]

Number

[0040] Thus, once the global local gradient vector g t is derived, Server 110 updates w t as shown in Equation 6.

[0041]

Number

[0042] Here, λ represents the learning rate.

[0043] The FL system repeats the federative learning process, including Equations 4-6, until the convergence condition is met or the maximum communication round T is reached.

[0044] This section describes the compression model for each device.

[0045] ≪Compression Model≫

[0046] According to one embodiment of the present invention, layer-scaled 1-bit quantization and layer-specific scarcity are used for efficient associative learning. Since aggressive gradient compression degrades convergence speed and learning performance, an error feedback mechanism is integrated to mitigate the negative effects of compression errors.

[0047] To implement this in an associative learning system, each device divides the local gradient vector into blocks (lower vectors) and derives a gradient size scaling element, which is the average of the parameters within each block, on a block-by-block basis. However, if the blocks consist of parameters from different layers, and the sizes of the parameters between layers differ significantly, the block size scaling element may not accurately reflect their importance.

[0048] Therefore, we treat the layers as blocks.

[0049] In each communication round t, each device compresses the error vector Local gradient vector compensated with TIFF2026093311000019.tif10128 added. Calculate TIFF2026093311000020.tif9134. This can be expressed by formula 7.

[0050]

number

[0051] Here, TIFF2026093311000022.tif9134 compensates for information loss due to gradient compression in the previous round. Then, each device is layer-specific size scaling element. Derive TIFF2026093311000023.tif9134 as shown in equation 8.

[0052]

number

[0053] Here, j∈{1,…J} i} indicates layer i.

[0054] Furthermore, each device uses a quantized local gradient vector. The TIFF2026093311000025.tif9134 is derived for each parameter as shown in equation 9.

[0055]

number

[0056] Here, j∈{1,…J} i} represents the parameter j of layer i. Therefore, compressed TIFF2026093311000027.tif9134 can be represented as shown in formula 10.

[0057]

number

[0058] Here, TIFF2026093311000029.tif10128 shows a rare masking indicator when layer i is not transmitted in each round t. TIFF2026093311000030.tif9128 Otherwise It will be shown as TIFF2026093311000031.tif8128.

[0059] Therefore, the compression error vector TIFF2026093311000032.tif9128 is updated as shown in formula 11.

[0060]

number

[0061] Here, TIFF2026093311000034.tif10130 is stored locally by each device k. Aggregating layers with small gradients only increases communication costs without significantly improving test accuracy. Therefore, transmitting only some layers with high gradients can reduce costs while maintaining test accuracy compared to transmitting all layers.

[0062] According to one embodiment of the present invention, layer-specific scarcity operates in a synchronized manner in which the same layer is scarce for all devices, thereby enabling the transmission of average gradient information from server 110 without loss of scarcity on the downlink. This reduces the downlink traffic size and improves the dropout effect in terms of information abstraction.

[0063] ≪Aggregation Model≫

[0064] Each device in each communication round t When TIFF2026093311000035.tif10130 is transmitted, server 110 aggregates it layer by layer, Derive TIFF2026093311000036.tif10130 as shown in formula 12.

[0065]

number

[0066] but, TIFF2026093311000038.tif10130 is transmitted using an analog method and aggregated in the air, therefore the lower vector of the signal received by the server TIFF2026093311000039.tif9133 is as shown in formula 13.

[0067]

number

[0068] Here, TIFF2026093311000041.tif9133 shows the channel coefficients between device k and the server, generated with N(0,1) and modeled with IID Rayleigh fading.

[0069] Assuming perfect upward link channel state information (CSI), the CSI of a block fading channel is changed independently across different rounds, without changing it in each round t.

[0070] Also, TIFF2026093311000042.tif9133 indicates the transmission power. TIFF2026093311000043.tif10140 shows the lower vectors of Additive White Gaussian Noise (AWGN). TIFF2026093311000044.tif11140 is Noise power σ 2 N(0,σ 2 It is generated by ).

[0071] To reduce signal distortion due to noise and fading, and to help Equation 13 become as close as possible to Equation 12, Design TIFF2026093311000045.tif9133 as shown in formula 14.

[0072]

number

[0073] Here, TIFF2026093311000047.tif9140 shows the (power) amplitude scaling factor. Therefore, each device must satisfy the transmission power constraint as shown in Equation 15.

[0074]

number

[0075] Here, P represents the maximum transmission power. TIFF2026093311000049.tif11128 is removed because its value is ±1.

[0076] Therefore, substituting equation 14 into equation 13, Redefine TIFF2026093311000050.tif8128 as shown in formula 16.

[0077]

number

[0078] Here, TIFF2026093311000052.tif9128 indicates preprocessing elements. Received Based on TIFF2026093311000053.tif8128, the server Estimate TIFF2026093311000054.tif10130 as shown in formula 17.

[0079]

number

[0080] Here, TIFF2026093311000056.tif10132 shows the post-processing elements. Server 110 to all devices Broadcast TIFF2026093311000057.tif8132 and update the parametric vector as shown in equation 18.

[0081]

number

[0082] All devices receive a perfect, distortion-free downlink channel through the high transmission power and bandwidth of Server 110, without any errors. Receive TIFF2026093311000059.tif8132.

[0083] For the analysis of the convergence rate, we make the following widely accepted assumptions about the compression operator, loss function, and gradient vector.

[0084] Assumption 1: Compression Operator

[0085] operator TIFF2026093311000060.tif8128 is an approximate factor as shown in equation 19. This shows the delta-approximation compression operator for TIFF2026093311000061.tif11156.

[0086]

number

[0087] Here, TIFF2026093311000063.tif10156 is converted into a layer-specific format and shown as in equation 20.

[0088]

number

[0089] Assumption 2: Loss function constraint

[0090] Global loss function The given parameter vector w for TIFF2026093311000065.tif9129 is the optimal parameter vector w as shown in equation 21. * The lower bound is defined by this.

[0091]

number

[0092] Assumption 3:

[0093] Global loss function The filename is TIFF2026093311000067.tif11137, and it is a gradient vector. TIFF2026093311000068.tif9137 represents the L-Lipschitz continuum.

[0094]

number

[0095] Here, L represents a non-negative constant, and equation 22 is expressed as in equation 23.

[0096]

number

[0097] Assumption 4: Gradient vector constraint

[0098] Local gradient vector g k This is the global gradient vector. These are independent and unbiased estimates of TIFF2026093311000071.tif10142.

[0099]

number

[0100] Here, G represents a positive constant, and when this is confirmed in layer-specific form, it is expressed as shown in Equation 25.

[0101]

number

[0102] Auxiliary arrangement 1:

[0103] To explain the compression error due to layer-scaled 1-bit quantization, layer-specific scarcity, and error feedback mechanism, we derive Auxiliary Arrangement 1, which defines the limit on the compression error based on Assumption 1 and Assumption 4, as shown in Equation 26.

[0104]

number

[0105] Here, S i teeth, The recent communication round t is TIFF2026093311000075.tif8128. Also, M i ≥ 1 is the maximum non-transmission period, and tS i ≦M i Therefore, according to equations 7, 11, and assumption 1, the compression error The limitations of TIFF2026093311000076.tif10130 are derived layer by layer as shown in equation 27.

[0106]

number

[0107] Since equation 27 has a repeating relationship, we derive equation 28 using the Peter-Paul inequality, the Jensen inequality, and assumption 4, where η > 0.

[0108]

number

[0109] To resolve the recurring relationship, all S from the past to the present i The memory m to save i ∈{0,…,S i Define} and m i Apply equations 27 and 28 to all communication rounds stored in the file to derive equation 29.

[0110]

number

[0111] Here, n∈{1,…,N} i} indicates the address, and η is, Set the filename to TIFF2026093311000080.tif12128.

[0112] Jensen's inequality allows us to rearrange the equation as shown in equation 30.

[0113]

number

[0114] Auxiliary arrangement 2:

[0115] In the associative learning system, to explain the aggregation error due to analog aggregation, we derive auxiliary calculation 2, which defines the aggregation error based on equation 27 and the distributed representation using expected values, as shown in equation 31.

[0116]

number

[0117] Auxiliary arrangement 3:

[0118] For convergence analysis of a rare 1-bit quantization aggregation method according to one embodiment of the present invention, the actual parametric vector w t This is an ideal parametric vector as shown in equation 32. We consider an uncompressed scenario defined as an approximation for TIFF2026093311000083.tif9134.

[0119]

number

[0120] Ideal parametric vector Auxiliary Calculation 3, which defines the update of TIFF2026093311000085.tif9134, is derived as shown in formula 33.

[0121]

number

[0122] Equation 33 is proven by sequentially inserting equations 18, 17, 11, and 7 into equation 32.

[0123]

number

[0124] Sorting 1:

[0125] Based on assumptions 1-4 and auxiliary adjustments 1-3, adjustment 1, which defines the convergence speed of the error feedback mechanism (referred to as SOBBA-EFO) based on rare 1-bit quantization aggregation according to one embodiment of the present invention, is derived as shown in equation 35.

[0126]

number

[0127] Based on assumption 3, in each communication round Global loss function for TIFF2026093311000089.tif8143 TIFF2026093311000090.tif9151 is restricted as shown in formula 36.

[0128]

number

[0129] Furthermore, under assumption 4, equation 36 is restricted to equation 37.

[0130]

number

[0131] The second term from the right side of equation 37 is restricted as shown in equation 38, based on the Peter-Paul inequality where ρ > 0.

[0132]

number

[0133] The second term from the right side of equation 38 is restricted to equation 39 by assumption 3 and auxiliary adjustment 1.

[0134]

number

[0135] Therefore, the second term from the right side of equation 37 is restricted as shown in equation 40.

[0136]

number

[0137] The last term from the right side of equation 37 is restricted to equation 41 by Jensen's inequality, assumption 4, and auxiliary adjustment 2.

[0138]

number

[0139] Insert equations 40 and 41 into equation 37 and rearrange them to form equation 42.

[0140]

number

[0141] By rearranging the terms according to assumption 2, we organize both mean values ​​for t as shown in equation 43.

[0142]

number

[0143] Here, in order to express it more simply without losing generality, if we set L and ρ to 1 / λ and 2-δ, respectively, we can rearrange it as shown in Equation 44.

[0144]

number

[0145] To demonstrate the effect of the error feedback mechanism, based on assumptions 1-4 and auxiliary adjustment 2, adjustment 2, which defines the convergence rate of the rare 1-bit quantization aggregation method (SOBBA-EFX) without an error feedback mechanism, is derived as shown in equation 45.

[0146]

number

[0147] Based on TIFF2026093311000101.tif9134, the loss function is derived as shown in Equation 46.

[0148]

number

[0149] By assumption 3, in each round t, w t+1 Global loss function TIFF2026093311000103.tif9129 is restricted as shown in formula 47.

[0150]

number

[0151] Furthermore, under assumption 4, equation 47 is restricted to equation 48.

[0152]

number

[0153] To express it more simply without sacrificing generality, we set L to 1 / λ. By assumption 1... Define TIFF2026093311000106.tif8147. Therefore, rearrange equation 48 again as equation 49, based on Auxiliary Arrangement 2 and Jensen's inequality.

[0154]

number

[0155] By assuming 3, we rearrange the terms and organize both mean values ​​for t as shown in equation 50.

[0156]

number

[0157] Observations of the first and second analyses suggest that associative learning systems can improve convergence performance by minimizing the total error upper limit.

[0158] The second term from the right of Equation 35 is the gradient weighted upper limit factor constant, while the other terms decrease as the maximum communication round T increases. In particular, the third and last terms represent the Weighted Aggregation Error (WAE) and Weighted Compression Error (WCE), respectively.

[0159] All terms on the right side of Equation 45 decrease as the maximum communication round T increases, and the second and last terms represent the weighted aggregation error (WAE) and weighted compression error (WCE), respectively. The weighted aggregation error (WAE), which is the negative effect of noise, is the amplitude scaling element. As TIFF2026093311000109.tif9140 increases, it decreases, scarce masking indicator TIFF2026093311000110.tif10128 is removed when it is 0, and reduced when it is 1. That is, When the filename is TIFF2026093311000111.tif9128, the weighted aggregation error (WAE) decreases while the weighted compression error (WCE) increases. The opposite is true when the filename is TIFF2026093311000112.tif8128. Therefore, TIFF2026093311000113.tif11155 defines weighted aggregation error (WAE) and weighted compression error (WCE), which are found to affect learning performance and communication cost. Using such theoretical convergence analysis... The compression ratio is determined through TIFF2026093311000114.tif10128. By controlling the transmission power through TIFF2026093311000115.tif9140, a collaborative optimization can be designed to minimize the convergence limit and total error.

[0160] I will explain this.

[0161] In the case of a rare 1-bit quantization aggregation scheme (SOBBA-EFO) with an error feedback mechanism, the server minimizes all amplitude scaling elements in each communication round t. TIFF2026093311000116.tif10134 and all rare masking indicators Define TIFF2026093311000117.tif10134.

[0162] After removing irrelevant terms in equation 35, the common elements are removed and the total error e t Define it as shown in equation 51.

[0163]

number

[0164] Formula 17 In order to prevent TIFF2026093311000119.tif10133 from being significantly distorted by noise, the post-processing noise must be reduced according to the expected value-based distributed representation, as shown in Equation 52.

[0165]

number

[0166] Here, TIFF2026093311000121.tif9133 shows the reduction rate. Observations of when TIFF2026093311000122.tif7133 approaches 0 indicate that formula 51 is When it is TIFF2026093311000123.tif8128 TIFF2026093311000124.tif is smaller than 9128. Therefore, all layers are transmitted in all rounds and all are negatively affected by noise.

[0167] In each communication round t, the communication cost of the federated learning system is defined as in Equation 53.

[0168]

Number

[0175] Therefore, Assumption 1, Auxiliary Arrangement 1, and By TIFF2026093311000129.tif11134 Restrict TIFF2026093311000130.tif9134 as shown in formula 55.

[0176]

number

[0177] Here, This shows the achievable upper limit for TIFF2026093311000132.tif10134. Using this, the transmission power constraint 15 is redefined as shown in equation 56.

[0178]

number

[0179]

number

[0180] Here, Defined using the remaining elements after excluding TIFF2026093311000135.tif12143.

[0181] To satisfy the new transmission power constraint (Equation 57), we assume that the maximum non-transmission period Mi is limited per layer by a positive constant, based on Equation 31 and t-Si=Mi. This can be expressed mathematically as shown in Equation 58.

[0182]

Number

[0183] Here, TIFF2026093311000137.tif8143 represents the maximum function, and M i is restricted as shown in Equation 59.

[0184]

Number

[0185] It satisfies Equation 57 based on Equation 59.

[0186] Based on Equation 54 and Equation 57, the co-optimization problem for the rare one-bit quantization aggregation method with an error mechanism (SOBAA-EFO) is formulated as shown in Equation 60.

[0187]

Number

[0191]

number

[0192] Here, This shows the achievable upper limit for TIFF2026093311000144.tif10148. Thus, the joint optimization problem for a rare 1-bit quantization aggregation method without error feedback is formulated as shown in Equation 63.

[0193]

number

[0194] Here, Instead of TIFF2026093311000146.tif11140 This is shown in TIFF2026093311000147.tif10140.

[0195] Real-valued variable amplitude scaling element TIFF2026093311000148.tif9140 and Binary Value Variable Scarcity Masking Indicator Due to its combination with TIFF2026093311000149.tif10128, P2 in equation 60 is a mixed-constant programming problem that is also non-convex. However, TIFF2026093311000150.tif10140 has no correlation between layers and between layers. Whatever TIFF2026093311000151.tif10128 is, P2 This means that the problem is transformed into a convex problem to determine the optimal solution for TIFF2026093311000152.tif9140.

[0196] Therefore, P2 is, We break down the file TIFF2026093311000153.tif10140 into two subproblems.

[0197] First, To determine TIFF2026093311000154.tif9140, we define an auxiliary function and then re-formulate a portion of P2 as shown in equation 64.

[0198]

number

[0199] all Since TIFF2026093311000156.tif9140 are uncorrelated, we decompose P4 in equation 64 into one smaller parallel subproblem. This can be expressed mathematically as shown in equation 65.

[0200]

number

[0201] To solve P4, we utilize the Lagrange dual method and multiply the constraint condition (equation 64b in equation 64) By introducing TIFF2026093311000158.tif10146, we derive the partial Lagrange as shown in equation 66.

[0202]

number

[0203] The double problem is derived as shown in equation 67.

[0204]

number

[0205] Here, we define the double function D(·) as shown in equation 68.

[0206]

number

[0207] Due to the strong duality between the primal problem and the dual problem, the primal problem P4 can be solved by solving the dual problem (P5 in equation 67). Thus, P5 can be decomposed into one smaller parallel convex subproblem, as shown in equation 69.

[0208]

number

[0209] Equation 69 can be solved using efficient convex optimization tools.

[0210] Next, Further auxiliary functions are defined to determine TIFF2026093311000163.tif10128, and a portion of P2 is formalized again as in equation 70.

[0211]

number

[0212] Here, TIFF2026093311000165.tif11147 shows the amplitude scaling element determined by resolving P5. P6 in equation 70 can be solved by an enumeration-based method that derives the best case from the number of all cases. However, for a rare 1-bit quantization aggregation method with an error feedback mechanism according to one embodiment of the present invention (SOBAA-EFO), this method is 2 I This is computationally impossible because it would require enumerating the number of all possible cases and then comparing them.

[0213] all Since TIFF2026093311000166.tif10128 are uncorrelated, we decompose P6 into I smaller parallel subproblems. This can be expressed mathematically as shown in Equation 71.

[0214]

number

[0215] Formula 60 for tS i =M i If so, the sub-problems This can be solved with TIFF2026093311000168.tif8128. Otherwise, each subproblem can be solved as shown in equations 72 and 73. The values ​​in TIFF2026093311000169.tif11140 must be compared to resolve the issue.

[0216]

number

[0217]

number

[0218] Since the constraint (60d in equation 60) may not be met, in order to minimize equation 70 while satisfying this constraint, equation 73 must be derived to find the layer with the smallest value. However, due to the difference in the number of parameters and gradient sizes between layers, this will always result in deriving only a specific layer in every round. Since each layer contributes differently to the learning model, deriving only a specific layer may reduce the convergence rate and learning performance.

[0219] Therefore, as in equation 74 TIFF2026093311000172.tif11140 has the smallest growth rate of layer i * We must derive this.

[0220]

number

[0221] Next, By setting TIFF2026093311000174.tif8128, all specified rare masking indicators are disabled. Obtain TIFF2026093311000175.tif10132. The computational complexity of solving smaller parallel subproblems is extended by I, so the computational complexity of the solutions for P5 and P6 are both The file is TIFF2026093311000176.tif9132. Therefore, the computational complexity of the solution according to one embodiment of the present invention, which sequentially resolves P5 and P6, is This is TIFF2026093311000177.tif9132, which is linearly extended by I. Thus, a solution according to one embodiment of the present invention determines the optimal value by computationally achievable complexity.

[0222] The pseudocode for a feasible solution of a disassembly substrate according to one embodiment of the present invention is shown in Figure 2. Here, TIFF2026093311000178.tif8132 is for all k and i This shows TIFF2026093311000179.tif12137.

[0223] In each round t, server 110 is b t,* and s t,* Derive the result and broadcast it to all devices. t,* and s t,* Given this, each device is given by equation 10 Derive TIFF2026093311000180.tif9128 and transmit it to the server using equation 16. To resolve P3 in equation 63, use equation 64 as an auxiliary function and define other auxiliary functions as shown in equation 75.

[0224]

number

[0225] The remaining derivation steps are identical to those for P2, and are therefore omitted.

[0226] For the sake of understanding and explanation, the operations performed by each device and server can be summarized again with reference to Figures 3 and 4 as follows:

[0227] Figure 3 is a flowchart showing an aerial federated learning method based on rare 1-bit quantization performed on each device according to one embodiment of the present invention.

[0228] In step 310, the device receives a global parametric vector from the server in each communication round t.

[0229] In step 315, the device derives a local gradient vector by applying a first-order approximation algorithm with the local data set based on the global parametric vector, compresses the local gradient vector by 1-bit quantization for each layer, and then scarces it. The device calculates the size-scaling elements of the layer-specific local gradient vector and derives a scarce 1-bit quantized local gradient vector using the calculated size-scaling elements and the layer-specific scarcity masking indicator.

[0230] In step 320, the device transmits the scarce 1-bit quantized local gradient vector to the server via the uplink channel in an analog manner.

[0231] Each stage corresponds to the same section explained in detail with reference to Figure 1, so overlapping explanations will be omitted.

[0232] Figure 4 is a flowchart showing an aerial federated learning method based on rare 1-bit quantization performed on each server according to one embodiment of the present invention.

[0233] In stage 410, the server determines and transmits the amplitude scaling element and binary scarcity masking indicator to each device based on the loss function and constraints for each communication round. This is the same as the section explained with reference to Figure 1, so the overlapping explanation will be omitted.

[0234] In step 415, the server receives signals from each device. Here, the received signals are scarce 1-bit quantized local gradient vectors.

[0235] In step 420, the server corrects the received signal using an amplitude scaling element and transmission power to reconstruct the gradient vector.

[0236] In step 425, the server aggregates the reconstructed gradient vectors, calculates and aggregates the global gradient vector, and updates the global parametric vector using the aggregated global gradient vector. Each step is equivalent to the part explained with reference to Figure 1, so overlapping explanations are omitted.

[0237] Considering an airborne federated learning system with a single server and K=25 devices, the maximum transmission power P and noise power σ are considered. 2 These are 10mW and 10 -4 We assumed mW.

[0238] To evaluate the learning performance and communication efficiency of rare 1-bit quantization aggregation (hereinafter referred to as SOBAA) according to one embodiment of the present invention, it was compared with OBDA (One-bit Broadband Digital Aggregation) and EFOBDA (Error-Feedback OBDA).

[0239] From the perspective of gradient compression systems, OBDA is a method that employs unscaled 1-bit quantization but does not employ scarcity and error feedback mechanisms. EFOBDA is a more advanced method than employing unscaled 1-bit quantization and error feedback mechanisms but does not employ scarcity. SOBAA-EFO / EFX according to one embodiment of the present invention employs layer-scaled 1-bit quantization and layer-specific scarcity, regardless of the presence or absence of an error feedback mechanism.

[0240] Using the well-known MNIST and CIFAR-10 datasets, we considered image classification, a fundamental task in vision recognition, as an application for experimental evaluation.

[0241] MNIST is a dataset consisting of 10 classes of 28x28 pixels and handwritten digits from 0 to 9 in black and white. It includes D=60,000 labeled training samples and 10,000 test samples. CIFAR-10 is a dataset consisting of 10 classes, each exclusively representing colorful objects or animals, with 32x32 pixels and RGB hues. It includes D=50,000 labeled training samples and 10,000 test samples.

[0242] In numerical experiments, two methods were considered for partitioning the data set across 25 devices.

[0243] 1) IID scheme for MNIST and CIFAR-10 data sets: The samples are shuffled and divided into 25 devices, and each device is D k =D / 25 samples were received.

[0244] 2) Non-IID method for the MNIST data set: After sorting the samples by label, they were divided into 200 fragments, each containing 300 samples, and assigned to 25 devices, with each device receiving 8 fragments.

[0245] For the IIDMNIST dataset, we implemented an MLP-based classifier with I=3 and J=199210.

[0246] This consisted of two hidden layers, each with 200 units, and each layer used the Rectified Linear Units (ReLU) activation function.

[0247] For the Non-IIDMNIST dataset, we implement a CNN-based classifier with I=4 and J=582026, denoted as 4NN.

[0248] This consisted of two 5x5 convolutional layers with ReLU activation functions, a fully connected layer with 512 units and ReLU activation, and a final softmax output layer. In the case of the convolutional layers, the first consisted of 32 channels and the second consisted of 64 channels, both connected by 2x2 max-pooling.

[0249] For the IIDCIFAR-10 dataset, we implement a CNN-based classifier with I=5 and J=940362, denoted as 5NN.

[0250] This consisted of two convolutional layers, two fully connected layers, and a linear transformation layer that generated a logit. All environments were implemented on a machine equipped with an NVIDIA Geforce RTX 4080, 32GB of memory, and an Intel(R) Core i9-13900K, using the Python programming language with the Pytorch library and the Matlab optimization toolbox. All methods (SOBAA, OBDA, EFOBDA) were evaluated in the same environment.

[0251] Regarding test accuracy, we observed the parameter at which all classifiers and all method combinations converged, setting the learning rate λ to 0.1 and the maximum communication round T for the MNIST and CIFAR-10 datasets to 200 and 500, respectively.

[0252] Preliminary experiments were conducted to determine the parameters for each classifier.

[0253] For example, Figure 5 shows the results of a preliminary experiment based on a dataset, specifically for the IIDMNIST dataset and MLP. First, error-free channels Assuming TIFF2026093311000182.tif9128, EFO and EFX do not perform layer-specific scarcity in all communication rounds. Based on Equation 25, in Figure 5(a), for communication round t Measure TIFF2026093311000183.tif9128 and determine the gradient bound. The file TIFF2026093311000184.tif9128 was set.

[0254] Based on equation 20, in Figure 5(b), for communication round t, The TIFF2026093311000185.tif9128 sample was measured, and the Approximation Factor δ was determined.

[0255] Based on equation 59, all M are determined by Figure 5(c). i The reduction rate is such that it becomes less than or equal to 10. The TIFF2026093311000186.tif7133 file was configured to transmit each layer at least once within 10 rounds.

[0256] Next, weighting coefficients θ were set for each device group and each network condition in the fading channel.

[0257] Figures 5(d) and 5(e) represent the communication costs required by Equation 53 to achieve the target test accuracy of 0.95. If the target test accuracy is not achieved, the communication cost is calculated as J × T. Regardless of the weighting coefficient θ, EFO was always lower than EFX due to the error feedback mechanism. As θ decreases, the first term in the RHS of both EFO and EFX increases in Equations 54 and 61, resulting in less layer transmission and lower communication costs. However, when θ is below a certain threshold, EFX fails to achieve the target test accuracy of 0.95, and the communication cost increases sharply.

[0258] In the airborne federation learning method, since all devices transmit simultaneously using the same time-frequency resources, the communication cost does not increase linearly with the number of devices; it is the same as the communication cost of a single device. However, as the number of devices increases, the number of training samples also increases, and the learning performance converges more quickly. Therefore, the number of communication rounds required for convergence decreases, and the communication cost decreases as shown in Figure 5(d).

[0259] Furthermore, the positive effect of increasing the number of training samples reduces the threshold of θ to mitigate the negative effect of rare transmissions in the layer. Similarly, Figure 5(e) shows that as noise power decreases, the signal becomes less distorted, leading to faster convergence of learning performance and reduced communication costs. Therefore, each combination (K,σ 2 To achieve the target test accuracy of 0.95 for ), we selected the most suitable θ to minimize communication costs.

[0260] To establish the parameters for the remaining classifiers and data sets, similar preliminary experiments were conducted before determining the parameters.

[0261] In Figure 5(f), the amplitude scaling element for the communication round t is calculated based on equations 8, 55, and 62. I measured TIFF2026093311000187.tif10132.

[0262] For both EFO and EFX, the size scaling elements transmitted in all communication rounds are consistently lower than estimated, indicating that the estimation work is well done.

[0263] Figure 6 shows the setting of experimental mediation parameters according to one embodiment of the present invention. The proposed SOBAA-EFO / EFX was evaluated for IIDMNIST and MLP, Non-IIDMNIST and 4NN, and IIDCIFAR-10 and 5NN, and its effects were demonstrated.

[0264] Figure 7 shows the evaluation of learning performance in error-free channels using the conventional and embodiment of the present invention, illustrating the maximum test accuracy, minimum training loss, and rank in error-free channels. Figure 8 shows the evaluation of learning performance in fading channels using the conventional and embodiment of the present invention, illustrating the test accuracy and training loss by communication round in fading channels.

[0265] For IIDMNIST and MLP, SOBAA-EFO achieved superior performance compared to SOBAA-EFX and EFOBDA in error-free channels. While SOBAA-EFO performed slightly lower than EFOBDA, this is because aggregating only the codes by the data set and classifier yielded better learning performance than aggregating both codes and sizes. However, the differences in test accuracy and training loss between SOBAA-EFO and EFOBDA were quite small, at 0.003 and 0.009, respectively.

[0266] As can be seen from Figures 8(a) and 8(d), SOBAA-EFO achieved the same ranking as error-free channels in fading channels. For Non-IIDMNIST and 4NN, SOBAA-EFO achieved the best learning performance in error-free channels, followed by EFOBDA, SOBAA-EFX, and OBDA in that order.

[0267] As can be seen from Figures 8(b) and 8(e), SOBAA-EFO achieved the same ranking as the error-free channel in the fading channel.

[0268] However, it can be seen that SOBAA-EFX exhibits a slightly higher training loss than OBDA due to layer-specific scarcity.

[0269] For IIDCIFAR-10 and 5NN, SOBAA-EFO achieved the best learning performance with error-free channels, followed by SOBAA-EFX, EFOBDA, and OBDA in test accuracy, and followed by EFOBDA, SOBAA-EFX, and OBDA in training loss.

[0270] As can be seen from Figures 8(c) and 8(f), SOBAA-EFX achieves the same rank as the error-free channel in terms of test accuracy on the fading channel, but is slightly higher than EFOBDA in terms of training loss due to layer-specific scarcity. Therefore, SOBAA shows better learning performance as the classifier and data set become more complex. In particular, for IIDCIFAR-10 and 5NN, SOBAA-EFX achieved better test accuracy than EFOBDA on both the error-free channel and the fading channel. The compression ratio was defined as shown in Equation 76 for each communication round t.

[0271]

number

[0272] Here, 32 represents the number of bits in the FloatTensor within the PyTorch library.

[0273] Figure 9 compares the communication efficiency of the conventional method and one embodiment of the present invention. Based on equation 76, Figures 9(a), 9(b), and 9(c) show that the proposed SOBAA, unlike existing methods, optimizes the gradient compression ratio by communication round. That is, unlike OBDA and EFOBDA, which show a fixed compression ratio of 1 / 32 for all classifiers and data sets, SOBAA shows different compression ratios depending on the communication round.

[0274] Next, SOBAA, proposed as a compromise between the target test accuracy and the communication cost required to achieve it, was compared with existing methods. If the target accuracy was not achieved, the communication cost was calculated as J × T. As shown in Figures 9(d), 9(e), and 9(f), the communication cost of OBDA increased sharply as the test accuracy increased, demonstrating the worst-case compromise. Furthermore, OBDA failed to achieve test accuracy of 0.95, 0.95, and 0.6 for MLP, 4NN, and 5NN, respectively.

[0275] EFOBDA showed a faster convergence rate than OBDA due to optimized error feedback mechanisms and power control, and presented an improved compromise with lower communication costs. SOBAA-EFX achieved a better compromise than EFOBDA except for the target test accuracy of 0.95 in MLP and 4NN, while SOBAA-EFO offered the best compromise.

[0276] SOBAA-EFO achieved communication costs that were 70.5% and 59.8% lower than EFOBDA, respectively, for MLP and 4NN, while maintaining a target test accuracy of 0.95. Furthermore, SOBAA-EFO achieved communication costs that were 59.6% lower than EFOBDA, while maintaining a target test accuracy of 0.7, for 5NN.

[0277] Figure 10 compares the effects of conventional and embodiment-1 of device populations, comparing the proposed SOBAA with other methods in terms of test accuracy with respect to the number of devices K, with the maximum communication round T set to 50. In all methods, test accuracy decreases as K decreases, because a smaller K results in a smaller number of training samples. In particular, when K falls below a certain threshold, test accuracy decreases geometrically rapidly.

[0278] Furthermore, the robustness of the method according to one embodiment of the present invention was demonstrated by comparing the test accuracy when K is 25 and when K is 5. When the number of devices K decreases from 25 to 5, the test accuracy of SOBAA-EFO proposed at IIDMNIST and MLP decreases by only 0.006, indicating that SOBAA-EFO is more robust than OBDA and SOBAA-EFX, but slightly less robust than EFOBDA.

[0279] The test accuracy of SOBAA-EFO proposed for Non-IIDMNIST and 4NN, as well as IIDCIFAR-10 and 5NN, decreases by only 0.01, indicating that SOBAA-EFO is more robust than OBDA, EFOBDA, and SOBAA-EFX.

[0280] Figure 11 compares the effects of network conditions under a conventional and embodiment of the present invention, with the maximum communication round K set to 50 and noise power σ 2 SOBAA, proposed in terms of test accuracy, was compared with other methods. All methods were σ 2 As σ increases, test accuracy decreases because signal distortion due to noise increases. In particular, σ 2 When a certain threshold was exceeded, the test accuracy decreased geometrically rapidly. Furthermore, in both SOBAA-EFO and SOBAA-EFX, while WCE remained largely unchanged, WAE increased proportionally, and convergence rate and learning performance declined. However, SOBAA according to one embodiment of the present invention mitigated this decline by adopting a suitable weighting coefficient θ. Also, σ2 10 -4 The robustness of the method according to one embodiment of the present invention was demonstrated by comparing the test accuracy when the value is and when it is 1.

[0281] σ 2 10 -4 When increasing from to 1, the test accuracy of SOBAA-EFX proposed by IIDMNIST and MLP decreases by only 0.0004, indicating that SOBAA-EFX is more robust than OBDA, EFOBDA, and SOBAA-EFO.

[0282] The test accuracy of SOBAA-EFO proposed for Non-IIDMNIST and 4NN decreases by only 0.003, indicating that SOBAA-EFO is more robust than OBDA and SOBAA-EFX, but slightly less robust than EFOBDA.

[0283] The test accuracy of SOBAA-EFO proposed in IIDCIFAR-10 and 5NN decreased by only 0.007, indicating that SOBAA-EFO is more robust than OBDA, EFOBDA, and SOBAA-EFX.

[0284] Figure 12 is a schematic diagram showing the internal configuration of a device or server according to one embodiment of the present invention.

[0285] A device or server according to one embodiment of the present invention includes, respectively, a communication unit 1210, a memory 1220, and a processor 1230.

[0286] The communication unit 1210 is a means for sending and receiving data with other devices via a communication network.

[0287] Memory 1220 stores a variety of instruction words for performing an airborne federated learning method based on rare 1-bit quantization according to one embodiment of the present invention.

[0288] The processor 1230 is a means for controlling internal components of a device or server (e.g., a communication unit 1210, a memory 1220, etc.) according to one embodiment of the present invention.

[0289] Furthermore, the processor 1230 executes the instruction words stored in memory 1220.

[0290] In the case of a device, the instruction word executed by the processor 1230 performs a series of processes: receiving a global parametric vector from the server in each communication round t; deriving a local gradient vector by applying a first-order approximation algorithm using the local data set based on the global parametric vector; compressing the local gradient vector by 1-bit quantization for each layer and then scarce it; and transmitting the scarce 1-bit quantized local gradient vector to the server via the uplink channel in an analog manner.

[0291] On the other hand, in the case of a server, the instruction words executed by the processor 1230 perform a series of processes in each communication round: determining amplitude scaling elements and binary scarcity masking indicators based on loss functions and constraints and transmitting them to each device; receiving signals from each device, which are scarce 1-bit quantized local gradient vectors; correcting the received signals with amplitude scaling elements and transmission power to reconstruct the gradient vectors; aggregating the reconstructed gradient vectors; calculating and aggregating the global gradient vectors; and updating the global parametric vectors using the aggregated global gradient vectors.

[0292] The apparatus and methods according to embodiments of the present invention are embodied in the form of program instructions performed by various computer means and recorded on a computer-readable recording medium. The computer-readable medium includes program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the computer-readable recording medium may be specifically designed and configured for the present invention, or may be publicly known and usable by those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, Proppy® disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magnetic-optical media such as floppy disks; and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine code generated by compilers, but also high-level language code executed by computers using interpreters, etc.

[0293] The hardware device described above is configured to operate with at least one software module to perform the operation of the present invention, and vice versa.

[0294] Although embodiments of the present invention have been described in detail above with reference to the drawings, the present invention is not limited to the embodiments described above, and can be modified and implemented in various ways without departing from the technical spirit of the present invention. [Explanation of Symbols]

[0295] 110 Servers 120 devices 1210 Communications Department 1220 memory 1230 processor

Claims

1. An aerial federated learning method based on rare 1-bit quantization of a computer-equipped device, The server receives global parametric vectors in each communication round, Based on the global parametric vector, a first-order approximation algorithm is applied using the local data set to derive a local gradient vector, and the local gradient vector is compressed by 1-bit quantization for each layer and then scarceed. The process includes the step of transmitting the reduced 1-bit quantized local gradient vector to the server via an uplink channel using an analog method. An aerial federation learning method based on rare 1-bit quantization, characterized in that the server aggregates and restores the rare 1-bit quantized local gradient vector, updates the global parametric vector, and then broadcasts it.

2. The step of compressing each layer by 1-bit quantization and then reducing its value is as follows: The step of calculating the size scaling elements of the local gradient vectors for each layer, A method for aerial federation learning based on rare one-bit quantization according to claim 1, comprising the step of deriving the rare one-bit quantization local gradient vector using the calculated size scaling elements and layer-specific scarcity masking indicators.

3. The layer-specific scarcity masking indicator is an indicator that determines whether a layer is transmitted, and is transmitted from the server in each communication round, as described in claim 2, for an aerial federated learning method based on scarcity 1-bit quantization.

4. The airborne federated learning method based on rare 1-bit quantization according to claim 2, characterized in that layers for which the scarcity masking indicator is 0 are excluded from transmission.

5. The aerial federated learning method based on rare 1-bit quantization according to claim 1, characterized in that the aforementioned layer is a layer constituting a deep learning model.

6. The method for learning an aerial federation based on rare 1-bit quantization according to claim 1, characterized in that deriving the local gradient vector includes calculating a local gradient vector corrected using the previous round compression error vector.

7. Updating the aforementioned global parametric vector means The analog signal received via the uplink is corrected by an amplitude scaling element and transmission power to reconstruct the gradient vector. The reconstructed gradient vectors are aggregated, and the global gradient vector is calculated and aggregated. The aerial federation learning method based on rare 1-bit quantization according to claim 1, characterized by including updating the global parametric vector using the aggregated global gradient vector.

8. The aerial federated learning method based on rare 1-bit quantization according to claim 7, further comprising the server transmitting the reconstructed gradient vector to the device.

9. An aerial federated learning method based on rare 1-bit quantization of a server equipped with a computer, (a) In each communication round, the steps include determining the amplitude scaling element and binary scarcity masking indicator based on the loss function and constraints and transmitting them to each device, (b) A step of receiving a signal from each device which is a scarce 1-bit quantized local gradient vector, (c) The step of correcting the received signal with an amplitude scaling element and transmission power to reconstruct the gradient vector, (d) A step of aggregating the reconstructed gradient vectors and calculating and aggregating the global gradient vector, (e) a step of updating a global parametric vector using the aggregated global gradient vector, characterized in that it is an aerial federation learning method based on rare 1-bit quantization.

10. The aerial federation learning method based on rare 1-bit quantization according to claim 9, characterized in that step (a) is performed in parallel to determine the amplitude scaling element and the binary rare masking indicator, and the binary rare masking indicator is determined after the amplitude scaling element is determined.

11. The airborne federated learning method based on rare 1-bit quantization according to claim 9, characterized in that the rare masking indicator is an indicator that determines whether or not the layer is transmitted.

12. A computer-readable recording medium having a program code for performing the method described in any one of claims 1 to 11.

13. Communications Department and, Memory to store at least one instruction word, The system comprises a processor that executes instruction words stored in the memory, When the instruction word is executed, the processor The server receives a global parametric vector in each communication round. Based on the global parameter vector, a first-order approximation algorithm is applied using the local data set to derive the local gradient vector, and after compressing the local gradient vector by 1-bit quantization for each layer, it is then scarce. The system is configured to transmit the reduced 1-bit quantized local gradient vector to the server via the uplink channel using an analog method. The server is a device characterized by aggregating and restoring the scarce 1-bit quantized local gradient vector, updating the global parametric vector, and then broadcasting it.

14. The aforementioned processor, After compressing each of the aforementioned layers by 1-bit quantization and then reducing its value, Calculate the size and scaling elements of the local gradient vectors for each layer. The device according to claim 13, characterized in that it is configured to derive the rare 1-bit quantized local gradient vector using the calculated size scaling elements and layer-specific scarcity masking indicators.

15. Communications Department and, Memory to store at least one instruction word, The system comprises a processor that executes instruction words stored in the memory, When the instruction word is executed, the processor (a) In each communication round, the amplitude scaling element and binary scarcity masking indicator are determined based on the loss function and constraints and transmitted to each device. (b) From each device, receive a signal which is a scarce 1-bit quantized local gradient vector, (c) The received signal is corrected by an amplitude scaling element and transmission power to reconstruct the gradient vector, (d) The reconstructed gradient vectors are aggregated, and the global gradient vector is calculated and aggregated, (e) A server characterized by being configured to update the global parametric vector using the aggregated global gradient vector.

Images