A Multi-Criterion User Selection Method for Wireless Hierarchical Federated Learning Systems

Abstract

The present invention discloses a multi-criterion user selection method for a wireless hierarchical federated learning system, which is executed by each edge server located in the middle layer of the wireless hierarchical federated learning system. Specifically, it includes: each edge server uses a multi-criterion user scoring mechanism based on fuzzy logic theory to obtain the fuzzy scores of all users within its coverage area, each edge server selects users according to the fuzzy score ranking, the users repeatedly selected by multiple edge servers reverse-select the edge servers, and the edge server selects substitute users, etc. The method of the present invention takes into account many factors such as the user's geographical location, the existing battery power, and the available computing resources, and can comprehensively consider the mutual influence among the above-mentioned various factors when selecting users, select users with stronger comprehensive capabilities to participate in federated learning, and thereby reduce the negative impact of the multi-dimensional heterogeneity of users on the performance of the federated learning system.

Description

Technical Field

[0001] The present invention belongs to the technical field of wireless communication, and particularly relates to a multi-criterion user selection method for a wireless hierarchical federated learning system. Background Art

[0002] Federated learning is a distributed machine learning model training architecture. Compared with the traditional centralized machine learning model training architecture, federated learning does not require all the original data to be aggregated to a central server. Instead, the original data is left at the intelligent terminals (hereinafter referred to as "users") where they are generated, and then the computing resources and data of each user are fully utilized for local model training. Each user collaborates to complete the global model training by sharing the locally trained model parameters. On the premise of completing the machine learning model training task, the federated learning technology can keep the original data generated at each user locally without sharing it with a third-party server, which protects the data privacy of each user. Secondly, since only effective information such as locally trained model parameters needs to be transmitted and a large amount of original data does not need to be transmitted, federated learning can effectively save limited communication resources and improve the data transmission and update efficiency. Hierarchical federated learning innovates the two-layer federated learning in terms of system architecture and algorithms. It fully absorbs the advantages of the two-layer federated learning architecture based on cloud servers or edge servers, can reduce the wireless communication time while covering a large range of users, and effectively improve the training performance of machine learning models.

[0003] Under the hierarchical federated learning architecture, the collaborative training of machine learning models requires a large number of users to participate through wireless communication. In actual application scenarios, there are significant differences among different users in multiple aspects such as geographical location, battery power, available computing resources, local data types, and data volume, that is, the multi-dimensional heterogeneity of users. This heterogeneity has a very large impact on the latency and energy consumption of the federated learning process and the convergence and accuracy of the trained model. Therefore, when the total number of users is much larger than the number of users participating in federated learning, the user selection scheme under the hierarchical federated learning architecture becomes particularly important.

[0004] In the prior art, in the random user selection scheme, each edge server randomly selects users within its own scope according to a predetermined number to participate in the federated learning model training task. The random selection scheme has the lowest execution complexity, but at the same time is the worst-performing scheme. This scheme cannot effectively screen according to the multi-dimensional heterogeneity of users to optimize the performance of federated learning; for another user selection scheme based on a single criterion, edge servers usually select users based on indicators such as distance, signal-to-noise ratio, and local data, or select users with the goal of maximizing system energy efficiency and minimizing model training delay. Such user selection schemes can achieve optimal performance in a certain sense of a criterion, but cannot effectively balance the heterogeneity of users in other dimensions, often resulting in performance losses in other aspects of federated learning. Summary of the Invention

[0005] In view of the above problems, the present invention provides a multi-criterion user selection method for a wireless hierarchical federated learning system, aiming to consider various factors such as user geographical location, existing battery power, and adjustable computing resources based on a multi-criterion user scoring mechanism based on fuzzy logic theory, so as to effectively address the performance damage of federated learning caused by the multi-dimensional heterogeneity of users.

[0006] The technical solution of the present invention is as follows:

[0007] A multi-criterion user selection method for a wireless hierarchical federated learning system includes the following steps:

[0008] Construct a wireless hierarchical federated learning system, which includes a cloud server, N edge servers, and M users. Each edge server can select K users within its coverage area to participate in federated learning;

[0009] Each edge server uses a multi-criterion user scoring mechanism based on fuzzy logic theory to obtain the fuzzy scores of all users within its coverage area, and selects users according to the fuzzy score ranking;

[0010] Among them, the multi-criterion user scoring mechanism based on fuzzy logic theory specifically includes:

[0011] Normalize the distance between user m and the corresponding edge server, user power, and user computing power respectively;

[0012] Input the normalized values into a fuzzifier for fuzzification processing to obtain fuzzy input sets corresponding to the distance between the user and the corresponding edge server, the user power, and the user computing power respectively;

[0013] Input the fuzzy input sets into a fuzzy inference engine, and the fuzzy inference engine performs fuzzy inference on the fuzzy input sets according to fuzzy rules to obtain a fuzzy output set;

[0014] The fuzzy output set is defuzzified by a defuzzifier to obtain the fuzzy score of user m.

[0015] A further technical solution of the present invention is that the normalization factors for normalizing the distance between user m and the corresponding edge server, the user's power, and the user's computing power are respectively the minimum distance among all users within the edge server range and the maximum values of the user's power and computing power.

[0016] A further technical solution of the present invention is that the fuzzy controller converts the normalized value into a corresponding fuzzy input set according to the membership function, and each element in the fuzzy input set contains a corresponding membership degree.

[0017] A further technical solution of the present invention is that the fuzzy rules of the fuzzy inference engine specifically include: rating according to the corresponding membership degrees contained in each element in the fuzzy input set, where the membership degree of the rating is the minimum value of the membership degrees of each element in the fuzzy input set, and combining the same ratings and their membership degrees to obtain a fuzzy output set.

[0018] A further technical solution of the present invention is that the defuzzification process of the defuzzifier adopts weighted average defuzzification, and the weight of each rating is the median of the non-zero normalized numerical output range corresponding to it.

[0019] A further technical solution of the present invention is that the edge server obtains the fuzzy scores of all users within its coverage area, sorts all users in descending order according to the fuzzy scores, and selects the top K users with the highest fuzzy scores to participate in the execution of the federated learning model training task.

[0020] A further technical solution of the present invention is that when a certain user is selected by multiple edge servers at the same time, the user chooses to join the edge server with the closest distance; the remaining edge servers then select the user with the highest fuzzy score in their respective remaining user sorting queues as a substitute until the number of selected users reaches K.

[0021] A further technical solution of the present invention is that the method further includes defining the normalized fitness υ of user m m for measuring the comprehensive ability of user m to execute the federated learning task, and the normalized fitness υ of user m m is the weighted sum of the normalized distance, normalized power, and normalized computing power, and the specific expression is:

[0022]

[0023] where ρ dis ,ρ bat and ρ cop are the weight factors of the user's normalized distance, power, and computing power respectively; dn,norm , e n,norm , f n,norm are the normalization factors of the distance, power, and computing power of user m and the corresponding edge server, respectively, and are set to the minimum value of the distances of all users within the range of the edge server, the maximum values of the user power and computing power, respectively. d n,m represents the distance d between user m and edge server n n,m , e m represents the power of user m, f m,max represents the computing power of user m.

[0024] A further technical solution of the present invention is that when normalizing the distance between user m and the corresponding edge server, the minimum value d of the distances of all users within the range of edge server n is used n,norm divided by the distance d between the user and edge server n n,m .

[0025] A multi-criterion user selection method for a wireless hierarchical federated learning system provided by the present invention takes into account many factors such as user geographical location, existing battery power, and mobilizable computing resources based on a multi-criterion user scoring mechanism based on fuzzy logic theory, and can comprehensively consider the mutual influence between the above-mentioned multiple factors when selecting users, select users with stronger comprehensive capabilities to participate in federated learning, and thus reduce the negative impact of the multi-dimensional heterogeneity of users on the performance of the federated learning system. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a schematic diagram of the structure of a wireless hierarchical federated learning system in an embodiment of the present invention;

[0027] Figure 2 is a schematic diagram of the process of a multi-criterion user selection method based on fuzzy logic in an embodiment of the present invention;

[0028] Figure 3 is a schematic diagram of a membership function in an embodiment of the present invention;

[0029] Figure 4 is an example diagram of a hierarchical federated learning scenario of a user selection method in an embodiment of the present invention;

[0030] Figure 5 is a schematic diagram of the influence of the number of users selected by the edge server on the average normalized fitness of users under different user selection schemes in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. In addition, it should be noted that for the convenience of description, only the parts related to the present invention rather than all the structures are shown in the drawings.

[0032] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts depict the steps as sequential processes, many of the steps can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the steps can be rearranged. The process can be terminated when its operations are completed, but it can also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0033] A multi-criterion user selection method for a wireless hierarchical federated learning system in an embodiment includes the following steps:

[0034] Construct a wireless hierarchical federated learning system, the system includes a cloud server, N edge servers and M users, and each edge server can select K users within its coverage area to participate in federated learning;

[0035] In a specific embodiment, the wireless hierarchical federated learning scenario is as Figure 1 shown, including a cloud server, N edge servers and M intelligent terminals (i.e., users). The cloud server, edge servers and intelligent terminals cooperate federatively to complete a machine learning model training task. The cloud server is connected to each edge server through a wired backhaul link, and the edge server is connected to the users within its service area through a wireless communication link.

[0036] Since the wireless communication resources of each edge server are limited, each edge server can only select K users within its range to participate in federated learning. In the embodiment, these K users are called active users, and the remaining unselected users are called dormant users. During the training process of the federated learning model, each user uses its own dataset to locally train the machine learning model, the edge server is responsible for aggregating the local model parameters of the users within its range into edge model parameters, and the cloud server is responsible for aggregating all edge model parameters into global model parameters.

[0037] Each edge server uses a multi-criterion user scoring mechanism based on fuzzy logic theory to obtain the fuzzy scores of all users within its coverage area, and selects users according to the fuzzy score ranking;

[0038] In a specific embodiment, refer to Figure 2As shown in the figure, in view of the performance degradation caused by the multi-dimensional heterogeneity of users in the wireless hierarchical federated learning system, a fuzzy logic-assisted multi-criterion user selection method is proposed, taking into account various factors such as the distance between the user and the edge server, the user's power, and the available computing resources. Each edge server should master the relevant information of all users within its coverage area. Before the model training task starts, each edge server uses the method proposed in the present invention to obtain the fuzzy scores of all users within its coverage area, selects users according to the fuzzy score ranking, and then starts training.

[0039] See Figure 2 , the multi-criterion user scoring mechanism based on fuzzy logic theory specifically includes:

[0040] Normalize the distance between user m and the corresponding edge server, the user's power, and the user's computing power respectively;

[0041] Furthermore, when normalizing the distance between user m and the corresponding edge server, the user's power, and the user's computing power respectively, the normalization factors are the minimum distance among all users within the edge server range and the maximum values of the user's power and computing power.

[0042] In a specific embodiment, the distance d between the user and its edge server n n,m , the user's power e m , the user's available computing resource f m,max (computing power) is normalized. The normalization factors are the minimum distance among all users within the edge server range, the maximum values of the power and computing power.

[0043] Input the normalized values into a fuzzifier for fuzzification to obtain the fuzzy input sets corresponding to the distance between the user and the corresponding edge server, the user's power, and the user's computing power respectively;

[0044] Furthermore, the fuzzifier converts the normalized values into corresponding fuzzy input sets according to the membership function, and each element in the fuzzy input set contains a corresponding membership degree.

[0045] In a specific embodiment, during the fuzzification process, the normalized values of the three criteria are input into the fuzzifier for fuzzification. The fuzzifier converts the input values of the three criteria into corresponding fuzzy input sets according to the membership functions shown in (a)-(c) in Figure 3 , and each element in it contains a corresponding membership degree. The fuzzy input sets corresponding to distance, power, and computing power are shown as follows:

[0046] Distance = {short, medium, long}

[0047] Power = {less, medium, more}

[0048] Computing power = {low, medium, high}

[0049] Input the fuzzy input set into a fuzzy inference engine, and the fuzzy inference engine performs fuzzy inference on the fuzzy input set according to fuzzy rules to obtain a fuzzy output set;

[0050] Furthermore, the fuzzy rules of the fuzzy inference engine specifically include: rating according to the corresponding membership degrees included in each element in the fuzzy input set, where the membership degree for rating is the minimum value of the membership degrees of each element in the fuzzy input set, and merging the same ratings and their membership degrees to obtain a fuzzy output set.

[0051] In a specific embodiment, during the fuzzy inference process, the fuzzy inference engine performs fuzzy inference on the fuzzy input sets of three criteria, namely distance, power, and computing power, according to the fuzzy rules shown in Table 1 to obtain a fuzzy output set, and the element type thereof can be represented as the following "rating" set. The membership degree of the rating corresponding to each fuzzy rule is the minimum value of the membership degrees of each element in the fuzzy input set corresponding to the rule. Merging the same ratings and their membership degrees, that is, obtaining a fuzzy output set.

[0052] Rating = {poor, relatively poor, medium, relatively good, good}

[0053] Table 1 Fuzzy Rules

[0054]

[0055]

[0056] Perform defuzzification processing on the fuzzy output set using a defuzzifier to obtain the fuzzy score of user m.

[0057] Furthermore, in the defuzzification process of the defuzzifier, weighted average defuzzification is adopted, and the weight of each rating is the median of its corresponding non-zero normalized numerical output range.

[0058] In a specific embodiment, during the defuzzification processing. The defuzzifier converts the fuzzy output set into the fuzzy score of this user m according to the membership function in (d) in Figure 3 . The defuzzification process adopts the weighted average defuzzification method, and the weight of each rating is the median of its corresponding non-zero normalized numerical output range.

[0059] Furthermore, the edge server obtains the fuzzy scores of all users within the coverage range, sorts all users in descending order according to the fuzzy scores, and selects the top K users with the highest fuzzy scores to participate in the execution of the federated learning model training task.

[0060] In a specific embodiment, at this point, the edge server successfully obtains the fuzzy score of its user m. The edge server repeats the above process to obtain the fuzzy scores of all users within its range, and sorts all users in descending order according to the scores. Finally, the edge server selects K users with the highest fuzzy scores to participate in the federated learning model training task.

[0061] Preferably, when a user is selected by multiple edge servers at the same time, the user chooses to join the nearest edge server; the remaining edge servers select the user with the highest fuzzy score as a substitute in their respective remaining user ranking queues until the number of users selected reaches K.

[0062] Furthermore, the method of the present invention further includes defining the normalized fitness υ of user m m It is used to measure the comprehensive ability of user m to perform federated learning tasks. The normalized fitness of user m is m It is the weighted sum of normalized distance, normalized power and normalized computing power. The specific expression is:

[0063]

[0064] Among them, ρ dis , ρ bat and ρ cop are the weight factors of the user's normalized distance, power and computing power respectively; d n,norm , e n,norm , f n,norm are the normalization factors of the distance between user m and the corresponding edge server, user power, and user computing power, respectively, and are set to the minimum value of the distance, user power, and user computing power of all users within the edge server range, respectively. n,m represents the distance d between user m and edge server n n,m , e m represents the power consumption of user m, f m,max Represents the computing power of user m.

[0065] Preferably, when normalizing the distance between user m and the corresponding edge server, the minimum value d of all user distances within the range of edge server n is used. n,norm Divide by the distance d between the user and the edge server n n,m .

[0066] To better demonstrate the effects of the present invention, the embodiments verified the multi-criterion user selection method of the present invention through simulation. The simulation experiment was conducted within a circular area with a radius of 1000 meters. A cloud server was located at the center of the circular area, and six edge servers were evenly distributed on a ring 500 meters away from the center of the circle. M users were evenly distributed in this circular area according to angles and radii. The available computing frequencies of the users were randomly distributed between 0 - 1 GHz, and the power of the users was randomly distributed between 0 - 1000 joules.

[0067] Figure 4 Fig. 4 is a scenario example diagram for implementing the user selection method proposed by the present invention in a wireless hierarchical federated learning system. In this scenario, the total number of users M = 500, and the number of users selected by each edge server K = 10. In Figure 4 Fig. 4, the five-pointed star at the center of the circular area represents the cloud server, the rhombus represents the edge server, the cross represents the users selected by the edge server, and the solid dots represent the unselected users. The ranges of each edge server are separated by solid lines. It can be observed that although some users are far from the edge servers, they are still selected because these users have relatively sufficient power and computing resources. This also demonstrates that the method proposed by the present invention selects users based on the comprehensive capabilities of the users.

[0068] Figure 5 Fig. 5 shows the variation of the average normalized fitness of users with the number of users selected by the edge server under different user selection schemes. It can be observed from the figure that the scheme proposed by the present invention is significantly better than the random user selection scheme and the user selection scheme based on computing resources for comparison, and can achieve performance gains of 70% and 16% respectively. For the same user selection scheme, the larger the number of candidate users M, the better the performance of the scheme, because the increase in the number of candidate users brings more users with strong comprehensive capabilities.

[0069] It can be seen from the embodiments that a multi-criterion user selection method for a wireless hierarchical federated learning system provided by the present invention, the multi-criterion user scoring mechanism based on fuzzy logic theory takes into account many factors such as the user's geographical location, the existing battery power, and the mobilizable computing resources, and can comprehensively consider the mutual influence among the above-mentioned various factors when selecting users, select users with stronger comprehensive capabilities to participate in federated learning, and thus reduce the negative impact of the multi-dimensional heterogeneity of users on the performance of the federated learning system.

[0070] In this article, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a step, method including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such a step, method.

[0071] The above content is a further detailed description of the present invention in combination with specific preferred embodiments. It cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A multi-criterion user selection method for a wireless hierarchical federated learning system, characterized in that It includes the following steps: Construct a wireless hierarchical federated learning system, which includes a cloud server, N edge servers, and M users. Each edge server can select K users within its coverage area to participate in federated learning; Each edge server uses a multi-criterion user scoring mechanism based on fuzzy logic theory to obtain the fuzzy scores of all users within its coverage area, and selects users according to the fuzzy score ranking; Among them, the multi-criterion user scoring mechanism based on fuzzy logic theory specifically includes: Normalize the distance between user m and the corresponding edge server, the user's power, and the user's computing power respectively; Input the normalized values into a fuzzifier for fuzzification to obtain the fuzzy input sets corresponding to the distance between the user and the corresponding edge server, the user's power, and the user's computing power respectively; Input the fuzzy input sets into a fuzzy inference engine, and the fuzzy inference engine performs fuzzy inference on the fuzzy input sets according to fuzzy rules to obtain a fuzzy output set; Use a defuzzifier to defuzzify the fuzzy output set to obtain the fuzzy score of user m; The method further includes defining a normalized fitness υ of user m m for measuring the comprehensive ability to perform a federated learning task on user m, and the normalized fitness υ of user m m is the weighted sum of the normalized distance, the normalized power, and the normalized computing power, and the specific expression is: Among them, ρ dis , ρ bat and ρ cop are the weight factors of the user's normalized distance, power consumption, and computing power respectively; d n,norm , e n,norm , f n,norm are the normalization factors of the distance between user m and the corresponding edge server, the user's power consumption, and the user's computing power, which are set to the minimum value of the distances of all users within the range of the edge server, the maximum value of the user's power consumption, and the user's computing power respectively. d n,m represents the distance d n,m between user m and edge server n, e m represents the power consumption of user m, and f m,max represents the computing power of user m.

2. The multi-criterion user selection method for a wireless hierarchical federated learning system according to claim 1, wherein The normalization factors for normalizing the distance between user m and the corresponding edge server, the user's power, and the user's computing power are respectively the minimum distance among all users within the edge server's range and the maximum values of the user's power and the user's computing power.

3. The multi-criterion user selection method for a wireless hierarchical federated learning system according to claim 1, wherein The fuzzifier converts the normalized values into corresponding fuzzy input sets according to the membership function, and each element in the fuzzy input set contains a corresponding membership degree.

4. The multi-criterion user selection method for a wireless hierarchical federated learning system according to claim 3, characterized in that The fuzzy rules of the fuzzy inference engine specifically include: rating according to the corresponding membership degrees contained in each element in the fuzzy input set, and the membership degree of the rating is the minimum of the membership degrees of each element in the fuzzy input set. Merge the same ratings and their membership degrees to obtain a fuzzy output set.

5. The multi-criterion user selection method for a wireless hierarchical federated learning system according to claim 4, wherein The defuzzification process of the defuzzifier uses weighted average defuzzification.

6. The multi-criterion user selection method for a wireless hierarchical federated learning system according to claim 1, characterized in that, The edge server obtains the fuzzy scores of all users within its coverage area, sorts all users in descending order according to the fuzzy scores, and selects the K users with the highest fuzzy scores to participate in the execution of the federated learning model training task.

7. The multi-criterion user selection method for a wireless hierarchical federated learning system according to claim 6, characterized in that, When a certain user is selected by multiple edge servers at the same time, the user chooses to join the edge server with the closest distance; the remaining edge servers then select the user with the highest fuzzy score in their respective remaining user ranking queues as a substitute until the number of selected users reaches K.

8. The multi-criterion user selection method for a wireless hierarchical federated learning system according to claim 1, wherein When normalizing the distance between user m and the corresponding edge server, the minimum distance d among all users within the range of edge server n is used n,norm Divided by the distance d between the user and edge server n n,m .

Images