Heterogeneous federal learning method and device based on structure perception information optimization and adaptive aggregation

Abstract

The invention discloses a heterogeneous federal learning method and device based on structure perception information optimization and adaptive aggregation, and the method comprises the steps: constructing a complex multi-scale model by a client, and carrying out the training of local data; obtaining structure sensing information of the multi-scale model, and forming a gradient adjustment factor; the client integrates the gradient adjustment factor into a local model for training, and extracts features of all data through a feature extractor; the client extracts more personalized feature representations and records corresponding labels, and after feature weighted averaging is conducted on the labels, the features and the corresponding labels are uploaded to the server; the server receives the average characteristics of each client and the corresponding label, and a global prediction head of the server is trained; the server broadcasts the updated global prediction header to the client, and each client prediction header adaptively aggregates the global prediction header based on local data; multiple rounds of training are repeated, and test indexes are calculated in each round; and each client selects an optimal model parameter, and inputs an image after importing to complete a classification task.

Description

Technical Field

[0001] The present invention belongs to the field of federated learning and privacy protection, and relates to a heterogeneous federated learning method and device based on structure-aware information optimization and adaptive aggregation. Background Art

[0002] With the growing demand for data privacy, federated learning, a machine learning method that can achieve collaborative model training without centralized data, is gradually becoming a research hotspot. The core idea of ​​federated learning is to train models locally on each client to ensure that the data always stays locally, and aggregate the trained models to a central server, thereby improving model performance while protecting data privacy. This feature makes federated learning show great application potential in highly privacy-sensitive fields such as medicine and finance. However, in actual applications, federated learning methods often face the practical challenges of data heterogeneity and model heterogeneity between clients. Data heterogeneity is mainly manifested in that the data of each client is not independent and identically distributed, while model heterogeneity refers to the fact that each client may adopt a different model architecture due to hardware, task requirements, etc. The challenges brought by these heterogeneities make it difficult for traditional federated learning methods such as FedAvg and FedProx to be directly applied to these complex scenarios, because they usually assume that all clients share the same model architecture and the data distribution is consistent.

[0003] To this end, a series of heterogeneous federated learning methods have emerged in recent years to address the challenges of data and model heterogeneity. However, existing heterogeneous federated learning methods still have some defects in dealing with model and data heterogeneity. Current heterogeneous model federated learning algorithms usually use existing simple general model architectures. However, the general model architecture cannot fully exploit the data characteristics unique to each client, but blindly using complex models will reduce the efficiency of federated learning training. Secondly, many existing heterogeneous federated learning methods fail to fully consider the personalized needs of clients. Due to the different data distribution and task requirements of each client, it is difficult to meet the performance requirements of all clients by replacing the local model with a server-side trained model. Summary of the invention

[0004] In view of the above-mentioned technical problems existing in the prior art, the present invention provides a heterogeneous federated learning method and device based on structure-aware information optimization and adaptive aggregation, aiming to improve the accuracy of federated learning in heterogeneous data and heterogeneous model environments.

[0005] The present invention adopts the following technical solution:

[0006] A first aspect of the present invention relates to a heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation, comprising the following steps:

[0007] Structure-aware federated learning feature enhancement stage:

[0008] Step 1: Build a complex multi-scale model and train it using local data;

[0009] Step 2: Obtain the structure-aware information of the multi-scale model in step 1, and use the structure-aware information to generate a gradient adjustment factor;

[0010] Step 3: Each client integrates the gradient adjustment factor into the local model for training. After the training is completed, each client extracts the features and labels of all data through the feature extractor;

[0011] Personalized adaptive aggregation stage based on autoencoder optimization:

[0012] Step 4: Each client uses the features and labels extracted in step 3 as a data set, and uses an autoencoder to extract more personalized feature representations and corresponding labels; each client performs a weighted average of the features of each label and uploads them to the server together with the corresponding label;

[0013] Step 5: The server receives the average features and corresponding labels of each client in step 4, and uses these data to train the server global prediction head;

[0014] Step 6: After the server-side global prediction header training is completed, the server broadcasts the updated global prediction header to the client; each client prediction header is based on local data and adaptively aggregates the global prediction header;

[0015] Step 7: Repeat steps 3 to 6 until the preset training rounds are reached, and calculate the test indicators in each round;

[0016] Step 8: Each client selects the optimal model parameters and imports the input image to complete the classification task.

[0017] Further, step 1 comprises:

[0018] Step 1-1: Build a complex multi-scale model. For the layers that need to improve the feature extraction capability locally, add an additional multi-scale feature enhancer MFE next to the original convolution, and add a scale regulator SC after the original convolution Conv and the newly added multi-scale feature enhancer MFE;

[0019] Step 1-2: After the multi-scale model is built, it is trained using local data.

[0020] Further, step 2 includes:

[0021] The scale regulator parameter sd of the trained multi-scale model, i.e., the structural perception information, is extracted, and the gradient adjustment factor GAF is obtained by calculation.

[0022] Further, step 3 includes:

[0023] Step 3-1: Incorporate the gradient adjustment factor into the local original model training and adjust the model's gradient update process to achieve a training process equivalent to that of the multi-scale model;

[0024] Step 3-2: After training, use the feature extractor For local datasets Extract features from each training sample and generate features .

[0025] Further, step 4 includes:

[0026] Step 4-1: Each client uses the encoder to generate local features from the local extractor Perform feature compression to remove redundant information and noise;

[0027] Step 4-2: Use the decoder to restore the compressed features to the original feature dimension size to ensure the dimensional consistency uploaded to the client;

[0028] Step 4-3: Each client calculates the average features of all samples belonging to the same class and uploads the average feature representation and corresponding labels to the server.

[0029] Further, step 5 comprises:

[0030] The server receives the average features and corresponding labels s from all participating clients and uses them as a dataset to train the server's global prediction head. It calculates the cross entropy loss between the prediction results of the server's global prediction head and the true category label s. It then uses the gradient descent method to adjust the parameters of the global prediction head. to update.

[0031] Furthermore, this step 6 includes:

[0032] The server broadcasts the server global prediction header after training. According to the local data of each client, the local client prediction header adaptively aggregates the downloaded server global prediction header.

[0033] Further, step 7 includes:

[0034] Repeat steps 3 to 6 until the preset training rounds are reached, and calculate the global average accuracy in each round.

[0035] The method of the present invention is divided into two stages. The first stage is the structure-aware federated learning feature enhancement stage, which aims to improve the feature extraction capability of the client feature extractor without increasing the model parameters. The second stage is the personalized adaptive aggregation stage based on autoencoder optimization, which aims to improve the client's prediction head personalization capability. The present invention improves the accuracy of heterogeneous federated learning by combining the structure-aware federated learning feature enhancement stage and the personalized adaptive aggregation stage based on autoencoder optimization.

[0036] The second aspect of the present invention relates to a heterogeneous federated learning device based on structure-aware information optimization and adaptive aggregation, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation of the present invention.

[0037] A third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation of the present invention.

[0038] The working principle of the present invention is: the present invention realizes the feature enhancement and personalized training of the client local model under the federated learning framework by combining structure-aware information optimization and adaptive aggregation technology. Specifically, firstly, through multi-scale model construction and training, the structure-aware information of the complex model is extracted, and the gradient adjustment factor is generated and integrated into the local training process of the client, thereby significantly improving the feature extraction ability of the local model. In addition, a more personalized feature representation is extracted through the autoencoder, and the personalized information of the client is uploaded to the server based on the weighted average of the features and labels for optimizing the global prediction head. After the global prediction head is updated, each client further optimizes the local model through adaptive aggregation to make it better adapt to the local data distribution.

[0039] The innovation of the present invention is: by constructing a complex multi-scale model, extracting the structural perception information of the model, and generating a gradient adjustment factor, the feature learning ability of the client local model is significantly enhanced, while avoiding the need to increase model parameters; an adaptive aggregation strategy is proposed: using an autoencoder to extract personalized features, and combining the dynamic weight aggregation method of the global prediction head and the local prediction head, personalized model optimization for heterogeneous data distribution is realized, which effectively improves the generalization ability and personalized performance of the client model.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. The present invention introduces a complex multi-scale model and extracts its structural perception information to generate a gradient adjustment factor, thereby effectively enhancing the feature extraction capability of the client's simple model. Without increasing the computational complexity, the present invention enables the simple model to accurately capture complex features, thereby improving the overall performance of federated learning.

[0042] 2. The present invention uses a personalized aggregation strategy based on autoencoders to enable each client to optimize its own feature representation according to local data characteristics. The autoencoder can compress high-dimensional features to remove redundant information, and restore key features through the decoder, thereby enhancing the expression and differentiation capabilities of the local model and significantly improving the personalized performance of the client. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the process of the present invention.

[0044] Figure 2 It is a schematic diagram of constructing a multi-scale model according to the method of the present invention.

[0045] Figure 3 It is a schematic diagram of the autoencoder feature compression and restoration of the method of the present invention.

[0046] Figure 4 It is a schematic diagram of client prediction header aggregation of the method of the present invention.

[0047] Figure 5 Schematic diagram of the device of the present invention. DETAILED DESCRIPTION

[0048] The specific implementation of the present invention is further described below in conjunction with specific embodiments:

[0049] Example 1

[0050] like Figure 1 As shown in the figure, a heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation is proposed, which includes the following steps:

[0051] Structure-aware federated learning feature enhancement stage:

[0052] Step 1: Build a complex multi-scale model and train it using local data;

[0053] Step 2: Obtain the structure-aware information of the multi-scale model in step 1, and use the structure-aware information to generate a gradient adjustment factor;

[0054] Step 3: Each client integrates the gradient adjustment factor into the local model for training. After the training is completed, each client extracts the features and labels of all data through the feature extractor.

[0055] Personalized adaptive aggregation stage based on autoencoder optimization:

[0056] Step 4: Each client uses the features and labels extracted in step 3 as a data set, and uses an autoencoder to extract more personalized feature representations and corresponding labels; each client performs a weighted average of the features of each label and uploads them to the server together with the corresponding label;

[0057] Step 5: The server receives the average features and corresponding labels of each client in step 4, and uses these data to train the server global prediction head;

[0058] Step 6: After the server-side global prediction header training is completed, the server broadcasts the updated global prediction header to the client. Each client prediction header is based on local data and adaptively aggregates the global prediction header;

[0059] Step 7: Repeat steps 3 to 6 until the preset training rounds are reached, and calculate the test indicators in each round;

[0060] Step 8: Each client selects the optimal model parameters and imports the input image to complete the classification task.

[0061] Step 1 specifically includes:

[0062] Step 1-1: In the structure-aware federated learning feature enhancement stage, such as Figure 2 , build a complex multi-scale model, add an additional multi-scale feature enhancer MFE next to the original convolution for the layers that need to improve the feature extraction capability locally, and add a scale regulator SC after the original convolution Conv and the newly added multi-scale feature enhancer MFE. The output formula after the input is constructed is as follows:

[0063] (1)

[0064] in, The jth layer represents the output after the construction layer, X represents the input data, represents the scaler of the j-th original convolutional layer, represents the output of X after the jth layer of original convolution, represents the output of X after the k-th multi-scale feature enhancer of the j-th layer, represents the kth scale regulator of the jth layer. This completes the construction of the multi-scale model. Then, the local data is used for training.

[0065] Step 1-2: After the multi-scale model is built, it is trained using local data to calculate the cross entropy loss between the multi-scale model prediction result and the true category label s. Then the gradient descent method is used to update the parameters of the multi-scale model.

[0066] Step 2 specifically includes:

[0067] After the multi-scale model training is completed, the scaler parameter sd, that is, the structural perception information, is extracted and the gradient adjustment factor GAF is generated. For the multi-branch layer, we can calculate the gradient adjustment factor GAF by the following formula:

[0068] (2)

[0069] in, represents the scaler parameters of the original convolution Conv, Represents the scaler parameters of the multi-scale feature enhancer MFE.

[0070] For a layer with only a single convolution branch, assume that its corresponding scaler parameter is , then the calculation formula of its GAF is:

[0071] (3)

[0072] in, Represents the scaler parameters of the convolutionConv layer with only a single convolution branch.

[0073] Step 3 specifically includes:

[0074] Step 3-1: Incorporate the gradient adjustment factor into the local original model training and adjust the model's gradient update process to achieve a training process equivalent to that of the multi-scale model. The formula for the gradient update after integration is:

[0075] (4)

[0076] in It is The model parameters after iterations are is the model parameter at the i-th iteration, is the learning rate, is the gradient adjustment factor, and L is the loss function. In the federated learning process, each client has its own local gradient adjustment factor.

[0077] Step 3-2: After training, use the feature extractor For local datasets Extract features from each training sample and generate features .

[0078] Step 4 specifically includes:

[0079] Step 4-1: If Figure 3, each client uses the encoder to generate local features from the local extractor Feature compression is performed to remove redundant information and noise. The compression process can be expressed as

[0080] (5)

[0081] in, represents the i-th compressed feature of the k-th client in the t-th round, represents the i-th unoptimized feature of the k-th client in the t-th round, Represents the encoding operation of the k-th client in the t-th round.

[0082] Step 4-2: If Figure 3 , the decoder restores the compressed features to the original feature dimension size to ensure the dimensional consistency uploaded to the client. The restoration process can be expressed as:

[0083] (6)

[0084] in, represents the i-th restored feature of the k-th client in the t-th round, represents the decoding operation of the kth client in the tth round, represents the i-th compressed feature of the k-th client in the t-th round.

[0085] Step 4-3: Each client calculates the average features of all samples belonging to the same class and uploads the average features and corresponding labels to the server. The formula for calculating the average features is as follows:

[0086] (7)

[0087] in, represents the average feature, Representation Class The client then uses these optimized average features Upload to the server.

[0088] Step 5 specifically includes:

[0089] The server receives the average features from all participating clients and the corresponding label s, and use this as the data set to train the server-side global prediction head. The server-side training round number is 100 rounds. Calculate the cross entropy loss between the prediction result of the server-side global prediction head and the true category label s. Then use the gradient descent method to adjust the parameters of the global prediction head. Update. The specific update formula is:

[0090] (8)

[0091] in, represents the global prediction head parameters, is the learning rate of the global prediction head, is the cross entropy loss function, represents the local average representation uploaded by the kth client in the tth round of federated learning training, and s is the corresponding label. The global prediction header on the server after training has all-category knowledge obtained from datasets across multiple clients, and has stronger generalization ability than the client prediction header that only contains local knowledge.

[0092] Step 6 specifically includes:

[0093] Each client receives the global prediction header from the server after training. According to the local data of each client, the local client prediction header adaptively aggregates the downloaded global prediction header from the server. and local prediction head The weights of the corresponding elements are adaptively aggregated to initialize the local prediction head. The aggregation calculation formula is:

[0094] (9)

[0095] in, represents the local prediction head parameters, Represents the clustering weight of each element. Each client uses the feature Iterative learning update The update formula is as follows

[0096] (10)

[0097] in, represents the clustering weight, is the learning rate used to learn the weights, are the local prediction head parameters, are the global prediction head parameters, Is the cross entropy loss function. Complete the update After that, the personalized parameters of the client prediction header can be calculated.

[0098] Step 7 specifically includes:

[0099] Repeat steps 3 to 6 until the preset number of training rounds is reached (the preset number of training rounds is 1000 rounds), record the model parameters of each client, and calculate the global average accuracy for each round. The formula is as follows:

[0100] (11)

[0101] in, represents the average accuracy, N represents the total number of images of all clients, Represents the number of images predicted correctly by the i-th client.

[0102] Step 8 specifically includes:

[0103] The model parameters of each client in the round with the best global average accuracy index are selected, the model parameters are imported into each client model, and the image classification is completed by inputting the image.

[0104] The specific data and scenario parameters of the present invention are described below. The present invention constructs two data heterogeneity scenarios: pathological setting and practical setting. In the pathological setting, we assign a dataset with non-overlapping categories and unbalanced data volume to each client. Each client obtains 2 / 10 / 10 / 20 categories of data from the CIFAR-10, CIFAR-100, Flowers102, and Tiny-ImageNet datasets, respectively. This setting simulates a heterogeneous training environment with significant differences in client data. In the practical setting, we use Dirichlet distribution to assign data to each client to simulate the data distribution in the real world. Specifically, a distribution ratio is extracted from the Dirichlet distribution. , the data set is classified into of proportion of data points assigned to client i.

[0105] To simulate heterogeneous federated learning scenarios, we set up an environment consisting of 20 clients with a client participation rate of 1. In order to simulate different federated learning scenarios, we adjusted the number and participation rate of clients in subsequent experiments. Each client data set is divided into a training set and a test set at a ratio of 3:1. Table 1 shows the accuracy comparison of the present invention and other methods on pathological setting data sets, and Table 2 shows the accuracy comparison of the present invention and other methods on practical setting data sets. The results show that the accuracy of the present invention is better than that of other methods.

[0106] Table 1 Performance comparison under pathological settings

[0107]

[0108] Table 2 Performance comparison under practical settings

[0109]

[0110] Example 2

[0111] This embodiment relates to a face recognition method based on the heterogeneous federated learning method of the present invention based on structure-aware information optimization and adaptive aggregation. Figure 1As shown, the steps include:

[0112] Step 1: Build a complex multi-scale model and train it using local data. Specifically:

[0113] Step 1-1: Each client prepares a face dataset and performs feature enhancement in the structure-aware federated learning stage, such as Figure 2 , build a complex multi-scale model, add an additional multi-scale feature enhancer MFE next to the original convolution for the layers that need to improve the feature extraction capability locally, and add a scale regulator SC after the original convolution Conv and the newly added multi-scale feature enhancer MFE. The output formula after the input is constructed is as follows:

[0114] (12)

[0115] in, The jth layer represents the output after the construction layer, X represents the input data, represents the scaler of the j-th original convolutional layer, represents the output of X after the jth layer of original convolution, represents the output of X after the k-th multi-scale feature enhancer of the j-th layer, represents the kth scale regulator of the jth layer. This completes the construction of the multi-scale model. Then, the local data is used for training.

[0116] Step 1-2: After the multi-scale model is built, it is trained using local data to calculate the cross entropy loss between the multi-scale model prediction result and the true category label s. Then the gradient descent method is used to update the parameters of the multi-scale model.

[0117] Step 2: Obtain the structure-aware information of the multi-scale model in step 1, and use the structure-aware information to generate the gradient adjustment factor. Specifically:

[0118] After the multi-scale model training is completed, the scaler parameters, that is, the structure perception information, are extracted and the gradient adjustment factor GAF is generated. For the multi-branch layer, we can calculate the gradient adjustment factor GAF by the following formula:

[0119] (13)

[0120] in, represents the scaler parameters of the original convolution Conv, Represents the scaler parameters of the multi-scale feature enhancer MFE.

[0121] For a layer with only a single convolution branch, assume that its corresponding scaler parameter is , then the calculation formula of its GAF is:

[0122] (14)

[0123] in, Represents the scaler parameters of the convolutionConv layer with only a single convolution branch.

[0124] Step 3: Each client integrates the gradient adjustment factor into the local model for training. After the training is completed, each client extracts the features and labels of all data through the feature extractor. Specifically:

[0125] Step 3-1: Incorporate the gradient adjustment factor into the local original model training and adjust the model's gradient update process to achieve a training process equivalent to that of the multi-scale model. The formula for the gradient update after integration is:

[0126] (15)

[0127] in It is The model parameters after iterations are The model parameters at the i-th iteration, is the learning rate, is the gradient adjustment factor, and L is the loss function. In the federated learning process, each client has its own local gradient adjustment factor.

[0128] Step 3-2: After training, use the feature extractor For local datasets Extract features from each training sample and generate features .

[0129] Step 4: Each client uses the features and labels extracted in step 3 as a data set, and uses an autoencoder to extract more personalized feature representations and corresponding labels; each client performs a weighted average of the features of each label and uploads them to the server together with the corresponding label. Specifically:

[0130] Step 4-1: If Figure 3 , each client uses the encoder to generate local features from the local extractor Feature compression is performed to remove redundant information and noise. The compression process can be expressed as

[0131] (16)

[0132] in, represents the i-th compressed feature of the k-th client in the t-th round, represents the i-th unoptimized feature of the k-th client in the t-th round, Represents the encoding operation of the kth client in the tth round.

[0133] Step 4-2: If Figure 3 , the decoder restores the compressed features to the original feature dimension size to ensure the dimensional consistency uploaded to the client. The restoration process can be expressed as:

[0134] (17)

[0135] in, represents the i-th restored feature of the k-th client in the t-th round, represents the decoding operation of the kth client in the tth round, represents the i-th compressed feature of the k-th client in the t-th round.

[0136] Step 4-3: Each client calculates the average features of all samples belonging to the same class and uploads the average features and corresponding labels to the server. The formula for calculating the average features is as follows:

[0137] (18)

[0138] in, represents the average feature, Representation Class The client then uses these optimized average features Upload to the server.

[0139] Step 5: The server receives the average features and corresponding labels of each client in step 4, and uses these data to train the server's global prediction head. Specifically:

[0140] The server receives the average features from all participating clients and the corresponding label s, and use this as the data set to train the server-side global prediction head. The server-side training round number is 100 rounds. Calculate the cross entropy loss between the prediction result of the server-side global prediction head and the true category label s. Then use the gradient descent method to adjust the parameters of the global prediction head. Update. The specific update formula is:

[0141] (19)

[0142] in, represents the global prediction head parameters, is the learning rate of the global prediction head, is the cross entropy loss function, represents the local average representation uploaded by the kth client in the tth round of federated learning training, and s is the corresponding label. The global prediction header on the server after training has all-category knowledge obtained from datasets across multiple clients, and has stronger generalization ability than the client prediction header that only contains local knowledge.

[0143] Step 6: After the server-side global prediction header training is completed, the server broadcasts the updated global prediction header to the client. Each client prediction header is based on local data and adaptively aggregates the global prediction header. Specifically:

[0144] Each client receives the global prediction header from the server after training, such as Figure 4 , based on each client’s local data, the local client prediction header adaptively aggregates the downloaded server-side global prediction header. We use the global prediction header and local prediction head The weights of the corresponding elements are adaptively aggregated to initialize the local prediction head. The aggregation calculation formula is:

[0145] (20)

[0146] in, represents the local prediction head parameters, Represents the clustering weight of each element. Each client uses the feature Iterative learning update The update formula is as follows

[0147] (twenty one)

[0148] in, represents the clustering weight, is the learning rate used to learn the weights, are the local prediction head parameters, are the global prediction head parameters, Is the cross entropy loss function. Complete the update After that, the personalized parameters of the client prediction header can be calculated.

[0149] Step 7: Repeat steps 3 to 6 until the preset training rounds are reached, and calculate the test indicators for each round. Specifically;

[0150] Repeat steps 3 to 6 until the preset number of training rounds is reached (the preset number of training rounds is 1000 rounds), record the model parameters of each client, and calculate the global average face recognition accuracy in each round. The formula is as follows:

[0151] (twenty two)

[0152] in, represents the average accuracy, N represents the total number of images of all clients, Represents the number of images predicted correctly by the i-th client.

[0153] Step 8: Each client selects the optimal model parameters, imports and inputs the face image to complete the face recognition task;

[0154] The model parameters of each client in the round with the best global average accuracy index are selected, the model parameters are imported into each client model, and the face recognition task is completed by inputting face images.

[0155] The following describes the specific data and scenario parameters of this embodiment. This embodiment constructs two data heterogeneity scenarios: pathological setting and practical setting. In the pathological setting, each client is assigned a data set with non-overlapping categories and unbalanced data volume. This setting simulates a heterogeneous training environment with significant differences in client data. In the practical setting, the Dirichlet distribution is used to allocate data to each client to simulate the data distribution in the real world. Specifically, a distribution ratio is extracted from the Dirichlet distribution, and the data of category c in the data set is divided into proportion of data points assigned to client i.

[0156] To simulate the heterogeneous federated learning scenario, this embodiment sets up an environment consisting of 10 clients with a client participation rate of 1. Each client dataset is divided into a training set and a test set at a ratio of 3:1. Table 3 shows the face recognition accuracy comparison between this embodiment and other methods on the MS-Celeb-1M face dataset with pathological settings and the MS-Celeb-1M face dataset with practical settings.

[0157] Table 3 Comparison of face recognition accuracy

[0158]

[0159] Example 3

[0160] Reference Figure 5 This embodiment relates to a heterogeneous federated learning device based on structure-aware information optimization and adaptive aggregation, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation of Example 1.

[0161] At the hardware level, the device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to the software implementation, the present invention does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0162] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs. The original code before compilation must also be written in a specific programming language, which is called Hardware Description Language (HDL). There is not only one hardware description language HDL, but many types, such as Advanced Boolean Expression Language ABEL (Advanced Boolean Expression Language), Altera Hardware Description Language AHDL (Altera Hardware Description Language), Confluence Hardware Description Language, Cornell University Programming Language CUPL (Cornell University Programming Language), HDCal, Java Hardware Description Language JHDL (Java Hardware Description Language), Hardware Description Language Lava, Hardware Description Language Lola, Hardware Description Language MyHDL, Hardware Description Language PALASM, Hardware Description Language Ruby Hardware Description Language RHDL (Ruby Hardware Description Language), etc. The most commonly used one is the Very-High-Speed ​​Integrated Circuit Hardware Description Language VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language Language) and the hardware description language Verilog.Those skilled in the art should also be aware that it is easy to obtain a hardware circuit that implements the logical method flow by simply performing some logic programming on the method flow using the above-mentioned hardware description languages ​​and programming it into an integrated circuit.

[0163] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: hardware description language ARC 625D, hardware description language Atmel AT91SAM, hardware description language MicrochipPIC18F26K20, and hardware description language Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices for implementing various functions included therein can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0164] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0165] For the convenience of description, the above device is described as being divided into various units according to their functions. Of course, when implementing the present invention, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0166] Example 4

[0167] This embodiment relates to a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation of Example 1 is implemented.

[0168] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0169] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0170] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0171] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0172] Each embodiment of the present invention is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0173] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation, characterized by: The following steps are involved: Step 1: Each client builds a complex multi-scale model and trains it using local data; Step 2: Obtain the structure-aware information of the multi-scale model in step 1, and use the structure-aware information to generate a gradient adjustment factor; Step 3: Each client integrates the gradient adjustment factor into the local model for training. After the training is completed, each client extracts the features of all data through the feature extractor; Step 4: Each client inputs the features extracted in step 3 into the autoencoder to extract a more personalized feature representation and record the corresponding label. Each client performs a weighted average of the features of each label and uploads them to the server together with the corresponding label. Step 5: The server receives the average features and corresponding labels of each client in step 4, and uses these data to train the server global prediction head; Step 6: After the server-side global prediction header training is completed, the server broadcasts the updated global prediction header to the client; each client prediction header is based on local data and adaptively aggregates the global prediction header; Step 7: Repeat steps 3 to 6 until the preset training rounds are reached, and calculate the test indicators in each round; Step 8: Each client selects the optimal model parameters and imports the input image to complete the classification task.

2. The heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation according to claim 1 is characterized in that: Step 1 includes: Step 1-1: Build a complex multi-scale model. For the layers that need to improve the feature extraction capability locally, add an additional multi-scale feature enhancer MFE next to the original convolution, and add a scale regulator SC after the original convolution Conv and the newly added multi-scale feature enhancer MFE; Step 1-2: After the multi-scale model is built, it is trained using local data.

3. The heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation according to claim 1 is characterized in that: Step 2 includes: The scale regulator parameter sd of the trained multi-scale model, i.e., the structural perception information, is extracted, and the gradient adjustment factor GAF is obtained by calculation.

4. The heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation according to claim 1 is characterized in that: Step 3 includes: Step 3-1: Incorporate the gradient adjustment factor into the local original model training and adjust the model's gradient update process to achieve a training process equivalent to that of the multi-scale model; Step 3-2: After training, use its feature extractor For local datasets Extract features from each training sample and generate features .

5. The heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation according to claim 1, characterized in that: Step 4 includes: Step 4-1: Each client uses the encoder to generate local features from the local extractor Perform feature compression to remove redundant information and noise; Step 4-2: Use the decoder to restore the compressed features to the original feature dimension size to ensure the dimensional consistency uploaded to the client; Step 4-3: Each client calculates the average features of all samples belonging to the same class and uploads the average feature representation and corresponding labels to the server.

6. The heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation according to claim 1, characterized in that: Step 5 includes: The server receives the average features and corresponding labels s from all participating clients and uses them as a dataset to train the server's global prediction head. It calculates the cross entropy loss between the prediction results of the server's global prediction head and the true category label s. It then uses the gradient descent method to adjust the parameters of the global prediction head. to update.

7. The heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation according to claim 1, characterized in that: Step 6 includes: The server broadcasts the server global prediction header after training. According to the local data of each client, the local client prediction header adaptively aggregates the downloaded server global prediction header.

8. The heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation according to claim 1, characterized in that: The test indicator described in step 7 is the global average accuracy.

9. A heterogeneous federated learning device based on structure-aware information optimization and adaptive aggregation, characterized in that: It includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that A program is stored thereon, and when the program is executed by a processor, the heterogeneous federated learning method based on structure-aware information optimization and adaptive aggregation as described in any one of claims 1 to 8 is implemented.

Images