Resource allocation optimization method and system for collaborative hybrid federated learning

Abstract

The present invention discloses a resource allocation optimization method for collaborative hybrid federated learning, comprising the following steps: S1. establishing a collaborative hybrid federated learning system for devices, users, and servers, calculating the similarity between device and user data, and determining the distribution of data between users and servers; S2. calculating the energy consumption and latency of devices, users, and servers at each stage of federated training in the collaborative hybrid federated learning system to obtain the system's energy consumption and latency; S3. determining the optimization variable bandwidth resource ratio and the number of communication rounds, and establishing a model with the goal of minimizing energy consumption using learning loss and communication latency as constraints; S4. converting the optimization problem into mathematical subproblems based on the model and optimization variable characteristics, solving the subproblems, and obtaining a solution to the resource allocation optimization problem. This method takes into account the heterogeneity of network device resources and data, effectively balances communication cost and model performance, and, based on the similarity criterion, makes the federated training node data more diverse, thereby improving the performance of model training.

Description

Technical Field

[0001] The present invention relates to the technical field of federated learning, and in particular to a resource allocation optimization method and system for collaborative hybrid federated learning. Background Art

[0002] Mobile devices have become the primary computing resource for billions of users worldwide, and billions more IoT devices are expected to come online in the coming years. These devices generate vast amounts of valuable data, and machine learning models trained using this data have the potential to improve the intelligence of many applications. However, enabling these capabilities on mobile devices often requires globally shared data on servers to train a satisfactory model. This can be undesirable from privacy, security, regulatory, or economic perspectives. Therefore, methods for storing data and sharing models on devices have become increasingly attractive.

[0003] Federated learning is a machine learning setting in which multiple "clients" (e.g., mobile phones) collaborate to train a model under the coordination of a central server. However, it still faces many challenges, among which client heterogeneity is a prominent issue. When clients have different computing and communication capabilities, some clients may not be able to complete their local training tasks in a timely manner. These clients become stragglers, hindering the overall training process.

[0004] To address this issue, active sampling techniques introduce a selection phase to select clients that meet certain requirements (such as computing power, communication bandwidth, data distribution, etc.). During training, only the selected clients participate in the actual training in each iteration. While it can alleviate the problem of falling behind to a certain extent, it limits the number of clients and thus misses training data. On the other hand, mobile cloud computing is an efficient and powerful technology that utilizes the abundant computing resources of remote cloud centers for big data processing. Therefore, in federated learning, devices with limited resources can fully utilize the resources of cloud servers to complete federated training. However, the transmission from device to cloud can cause severe congestion and communication delay. Due to the long transmission distance from device to server, mobile cloud computing cannot meet the critical latency requirements.

[0005] Fortunately, relays can regenerate and transmit received signals, thereby increasing the distance the signal can be transmitted. In addition, mobile edge computing can perform calculations on the received data. Mobile edge servers are deployed at the edge of the network, such as base stations, access points, etc., to provide cloud computing services to mobile devices. Future wireless networks hope to serve a wide range of devices with heterogeneous communication or computing resources, so that some resource-constrained devices may not be able to perform tasks in a timely manner, thereby reducing the performance of the entire system. To solve this problem, a collaborative approach to explore and allocate computing resources of devices in the network is an effective method. This paradigm is called collaborative computing, and its core idea is that "strong" devices (with abundant communication or computing resources) can assist adjacent "weak" devices. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings and deficiencies of existing federated learning technical solutions and provide a resource allocation optimization method for collaborative hybrid federated learning. By optimizing the bandwidth resource ratio and the number of communication rounds, it aims to minimize energy consumption under the constraints of learning loss and delay, effectively balancing communication cost and model performance.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] A resource allocation optimization method for collaborative hybrid federated learning includes the following steps:

[0009] S1. Establish a collaborative hybrid federated learning system for devices, users, and servers to calculate the similarity between device and user data and determine the distribution of data between users and servers.

[0010] S2. Calculate the energy consumption and latency of devices, users, and servers at each stage of federated training in the collaborative hybrid federated learning system to obtain the system's energy consumption and latency.

[0011] S3. Determine the optimized variable bandwidth resource ratio and the number of communication rounds, and establish a model with the goal of minimizing energy consumption, using learning loss and communication latency as constraints.

[0012] S4. Convert the optimization problem into mathematical sub-problems based on the model and optimization variable characteristics, solve the sub-problems, and obtain a solution to the resource allocation optimization problem.

[0013] Preferably, the data similarity between the device and the user is calculated, and the distribution of the data between the user and the server is determined, specifically:

[0014] The cosine similarity is used to calculate the data similarity between the device and the user. A similarity threshold is set. If the data similarity between the device and the user is higher than the similarity threshold, the data of the device and the user are considered similar. Otherwise, the data of the device and the user are considered dissimilar.

[0015] If the data of the device and the user are similar, the user relays the device data to the server for training. If the data of the device and the user are not similar, the user uses the device data for user local training.

[0016] Preferably, the collaborative hybrid federated learning system performs federated training, which includes the following stages:

[0017] Phase 1: Each device offloads data to the corresponding service user;

[0018] Phase 2: After each user receives the device data for the corresponding service, it first determines whether the device data is similar to the user's data. If the device data is similar, the device data is directly forwarded to the server;

[0019] If the device data is not similar to the user's, the device data is added to the user's local data for local training to obtain a local model, and then the user transmits the local model to the server;

[0020] Phase 3: The server uses the user's transmitted data for local training to obtain a local model. The server aggregates the trained local model and the local model sent by the user into a global model.

[0021] Preferably, the energy consumption and latency of the device are calculated by:

[0022] In the first phase, the achievable data rate for the device to offload data to the user is:

[0023] R i′ =β i′ C i′

[0024] Where, β i′ is the bandwidth ratio allocated to device i′, C i′ is the channel capacity of device i′, R i′ The achievable data rate for device i′ to offload data to user i;

[0025] The channel capacity calculation formula of the device is:

[0026]

[0027] Where B is the total bandwidth, p i′ is the transmission power of device i′, h i′,i is the channel power gain from device i′ to user i, n0 is the background noise power;

[0028] The time consumed by the device to transmit data is:

[0029]

[0030] Where D i′ is the local data of device i′, The time for device i′ to transmit data;

[0031] The energy consumed by the device to transmit data is:

[0032]

[0033] Where, Energy consumption of transmitting data to device i′.

[0034] Preferably, the energy consumption and latency of the user are calculated by:

[0035] In the second stage, users offload similar data to the server, and the achievable data rate for transmitting data is:

[0036]

[0037] Where, The bandwidth ratio allocated to user i when offloading similar data to the server; C i is the channel capacity of user i; The achievable data rate for offloading similar data to the server for user i;

[0038] The calculation formula for the user's channel capacity is:

[0039]

[0040] Where p i is the transmission power of user i, h i is the channel power gain from user i to the server;

[0041] The user offloads similar data to the server, and the time consumed for data transmission is:

[0042]

[0043] Where, Represents the device set served by user i; represents a set of devices that are dissimilar to the data of user i,

[0044] The user offloads similar data to the server, and the energy consumed in transmitting data is:

[0045]

[0046] Where, The energy consumed in transmitting data for user i;

[0047] In the second stage, the user offloads the local model to the server, and the achievable data rate of the transmitted data is:

[0048]

[0049] in, The bandwidth ratio allocated to user i when the user offloads the local model to the server; R i model The achievable data rate for offloading the local model to the server for user i;

[0050] The user unloads the local model to the server, and the time consumed for data transmission is:

[0051]

[0052] Where W is the size of the local model in bits; i model The time consumed in transmitting the local model for user i;

[0053] The user offloads the local model to the server, and the energy consumed by transmitting data is:

[0054] e i model =p i t i model

[0055] Where, e i model Energy consumed by transmitting the local model for user i;

[0056] For user local training, the time consumed by each user is:

[0057]

[0058] Where D i is the local data of user i, c is the number of CPU cycles in model training, t i comp The time consumed by user i for local training, f i is the CPU cycle speed of user i;

[0059] The energy consumed by each user is:

[0060]

[0061] Where ξ is the chip capacitance related to the CPU, e i comp The energy consumed by local training for user i.

[0062] Preferably, the energy consumption and latency of the server are calculated by:

[0063] In the third stage, the time consumed by the server training model is:

[0064]

[0065] Where, represents the user set; f I+1 is the CPU cycle speed of the server; The time consumed by the server to train the model; the energy consumed by the server to train the model is:

[0066]

[0067] Where, e I+1 Energy consumed by the server to train the model;

[0068] The total delay of each round of communication in the system is:

[0069] T=max{T1,T2}

[0070] Where T1 is the time for data transmission from device to user, from user to server, and for server calculation; T2 is the time for data transmission from device to user, from user calculation, and from user to server local model; and T is the total delay of each round of communication in the system.

[0071] The calculation formulas for T1 and T2 are:

[0072]

[0073] Where γ is the number of local training iterations in each round of communication;

[0074] The total energy consumed by the system in the communication link is obtained as:

[0075]

[0076] Where E is the total energy consumed by the system in the communication link.

[0077] Preferably, the model established with the goal of minimizing energy consumption is:

[0078]

[0079] Where β is the allocation ratio of bandwidth resources, Unloading similar data to the server for users, allocating bandwidth resources in proportion, The bandwidth allocation ratio for users to offload local models to the server. K is the total number of communication rounds;

[0080] The constraints of the model are:

[0081] F(w)≤ζ

[0082] KT≤T max

[0083]

[0084]

[0085]

[0086]

[0087] Where, is the maximum allowed learning loss, T max is the maximum allowed delay for the entire federation training, Is a positive integer.

[0088] A collaborative hybrid federated learning system, applied to any of the above-mentioned collaborative hybrid federated learning resource allocation optimization methods, comprising:

[0089] One server, multiple users, and multiple devices. Multiple users and one server form the edge layer, and multiple devices form the perception layer. Each user serves a group of disconnected devices, and each user and each device stores local data.

[0090] Each user is used to obtain data from the corresponding service device, determine the data similarity between the device and the user, add the device data to the local data for user local training, or send the device data to the server for training through the user relay.

[0091] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the resource allocation optimization method for collaborative hybrid federated learning described in any one of the above items are implemented.

[0092] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned resource allocation optimization methods for collaborative hybrid federated learning.

[0093] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0094] 1. Taking into account the heterogeneous characteristics of network device resources and data, this paper designs a hierarchical collaborative hybrid federated learning framework, and for the first time proposes a scenario in which each user plays the dual role of relay and server computing at the same time.

[0095] 2. Based on the proposed collaborative hybrid federated learning framework, this paper proposes an energy consumption minimization problem under the constraints of learning loss and delay by optimizing bandwidth and number of communication rounds, effectively balancing the communication cost and the accuracy of the federated learning model.

[0096] 3. The present invention is simple and effective, and the performance of the proposed framework is higher than that of traditional federated learning. The present invention only requires a user-server interaction model, and the total amount of interaction is small. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] Figure 1 Flowchart of the resource allocation optimization method for collaborative hybrid federated learning.

[0098] Figure 2 Schematic diagram of the structure of the collaborative hybrid federated learning system.

[0099] Figure 3 Schematic diagram of the federated training process of the collaborative hybrid federated learning system. DETAILED DESCRIPTION

[0100] The resource allocation optimization method and system for collaborative hybrid federated learning of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0101] See also Figure 1 The present invention discloses a resource allocation optimization method for collaborative hybrid federated learning, which includes the following steps:

[0102] S1. Establish a collaborative hybrid federated learning system for devices, users, and servers to calculate the similarity between device and user data and determine the distribution of data between users and servers.

[0103] S2. Calculate the energy consumption and latency of devices, users, and servers at each stage of federated training in the collaborative hybrid federated learning system to obtain the system's energy consumption and latency.

[0104] S3. Determine the optimized variable bandwidth resource ratio and the number of communication rounds, and establish a model with the goal of minimizing energy consumption, using learning loss and communication latency as constraints.

[0105] S4. Convert the optimization problem into mathematical sub-problems based on the model and optimization variable characteristics, solve the sub-problems, and obtain a solution to the resource allocation optimization problem.

[0106] Communication networks have the following two characteristics: network nodes are heterogeneous in terms of communication or computing conditions, that is, the heterogeneity of network resources; and the data distribution of adjacent devices may have significant overlap, that is, the heterogeneity of data. The present invention mainly studies federated learning on hierarchical collaborative computing networks based on the above two characteristics of communication networks. Specifically, the device offloads the perceived data to the edge layer for federated model training. For the perception data received by the user from the device, based on the data similarity comparison, it is either used for local training of the user or forwarded to the server through the user. That is, each user plays the dual role of relay and server computing at the same time to solve the problem of heterogeneity of data and resources.

[0107] In view of the heterogeneity of network resources, the present invention adopts data similarity rules to distribute perception data between users and servers. Taking into account the problems of long-distance transmission and limited computing resources, the present invention combines the concepts of relay and mobile edge computing, allowing users to assume the dual identities of relay and mobile edge server, and designs a hierarchical collaborative hybrid federated learning framework composed of perception devices, users, and servers. The framework consists of a perception layer with multiple devices and an edge layer with multiple users and a central server. Based on the proposed framework, the bandwidth resource ratio of devices and users and the number of communication iterations (number of communication rounds) between users and servers are optimized. Under the constraints of learning loss and delay, a model with the goal of minimizing energy consumption is established, which effectively balances communication cost and model performance.

[0108] like Figure 2 As shown, the collaborative hybrid federated learning system includes a server, multiple users and multiple devices. Multiple users and a server form an edge layer, multiple devices form a perception layer, each user serves a group of unconnected devices, and each user and each device stores local data. Each user is used to obtain the data of the corresponding service device, and to determine the data similarity between the device and the user, and to add the device data to the local data for local training of the user, or to send the device data to the server for training through the user relay. The collaborative hybrid federated learning system of the present invention can execute the resource allocation optimization method of the collaborative hybrid federated learning of the present invention, and can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0109] Represents a user set, Represents the set of devices served by user i. For convenience, we use i′ to represent the device served by user i. Each device i′ has local data Each user i has local data Local data of device i′ The cardinality is expressed as Local data of user i The cardinality is expressed as

[0110] In step S1, the data similarity between the device and the user is calculated to determine the distribution of data between the user and the server. Specifically,

[0111] The cosine similarity is used to calculate the data similarity between the device and the user. A similarity threshold is set. If the data similarity between the device and the user is higher than the similarity threshold, the data of the device and the user are considered similar. Otherwise, the data of the device and the user are considered dissimilar.

[0112] If the data of the device and the user are similar, the user relays the device data to the server for training. If the data of the device and the user are not similar, the user uses the device data for user local training.

[0113] In step S2, if Figure 3 As shown in Figure 2, the collaborative hybrid federated learning system performs federated training, which includes the following stages:

[0114] Phase 1: Each device offloads data to the corresponding service user;

[0115] Phase 2: After each user receives the device data for the corresponding service, it first determines whether the device data is similar to the user's data. If the device data is similar, the device data is directly forwarded to the server;

[0116] If the device data is not similar to the user's, the device data is added to the user's local data for local training to obtain a local model, and then the user transmits the local model to the server;

[0117] Phase 3: The server uses the user's transmitted data for local training to obtain a local model. The server aggregates the trained local model and the local model sent by the user into a global model.

[0118] The energy consumption and latency of the device are calculated using the following method:

[0119] In the first phase, the achievable data rate for the device to offload data to the user is:

[0120] R i′ =β i′ C i′

[0121] Where, β i′ is the bandwidth ratio allocated to device i′, C i′ is the channel capacity of device i′, R i′ The achievable data rate for device i′ to offload data to user i;

[0122] The channel capacity calculation formula of the device is:

[0123]

[0124] Where B is the total bandwidth, p i′ is the transmission power of device i′, h i′,i is the channel power gain from device i′ to user i, n0 is the background noise power;

[0125] The time consumed by the device to transmit data is:

[0126]

[0127] Where D i′ is the local data of device i′, The time for device i′ to transmit data;

[0128] The energy consumed by the device to transmit data is:

[0129]

[0130] Where, Energy consumption of transmitting data to device i′.

[0131] The energy consumption and latency of the user are calculated as follows:

[0132] In the second stage, users offload similar data to the server, and the achievable data rate for transmitting data is:

[0133]

[0134] Where, The bandwidth ratio allocated to user i when offloading similar data to the server; C i is the channel capacity of user i; The achievable data rate for offloading similar data to the server for user i;

[0135] The calculation formula for the user's channel capacity is:

[0136]

[0137] Where p i is the transmission power of user i, h i is the channel power gain from user i to the server;

[0138] The user offloads similar data to the server, and the time consumed for data transmission is:

[0139]

[0140] Where, Represents the device set served by user i; represents a set of devices that are dissimilar to the data of user i,

[0141] The user offloads similar data to the server, and the energy consumed in transmitting data is:

[0142]

[0143] Where, The energy consumed in transmitting data for user i;

[0144] In the second stage, the user offloads the local model to the server, and the achievable data rate of the transmitted data is:

[0145]

[0146] in, The bandwidth ratio allocated to user i when offloading the local model to the server; The achievable data rate for offloading the local model to the server for user i;

[0147] The user unloads the local model to the server, and the time consumed for data transmission is:

[0148]

[0149] Where W is the size of the local model in bits; The time consumed in transmitting the local model for user i;

[0150] The user offloads the local model to the server, and the energy consumed by transmitting data is:

[0151]

[0152] Where, Energy consumed by transmitting the local model for user i;

[0153] For user local training, the time consumed by each user is:

[0154]

[0155] Where D i is the local data of user i, c is the number of CPU cycles in model training, The time consumed by user i for local training, f i is the CPU cycle speed of user i;

[0156] The energy consumed by each user is:

[0157]

[0158] Where ξ is the chip capacitance related to the CPU, The energy consumed by local training for user i.

[0159] The energy consumption and latency of the server are calculated as follows:

[0160] In the third stage, the time consumed by the server training model is:

[0161]

[0162] Where, represents the user set; f I+1 is the CPU cycle speed of the server; The time consumed by the server to train the model; the energy consumed by the server to train the model is:

[0163]

[0164] Where, e I+1 Energy consumed by the server to train the model;

[0165] The total delay of each round of communication in the system is:

[0166] T=max{T1,T2}

[0167] Where T1 is the time for data transmission from device to user, from user to server, and for server calculation; T2 is the time for data transmission from device to user, from user calculation, and from user to server local model; and T is the total delay of each round of communication in the system.

[0168] The calculation formulas for T1 and T2 are:

[0169]

[0170] Where γ is the number of local training iterations in each round of communication;

[0171] The total energy consumed by the system in the communication link is obtained as:

[0172]

[0173] Where E is the total energy consumed by the system in the communication link.

[0174] In step S3, the model established with the goal of minimizing energy consumption is:

[0175]

[0176] Where β is the allocation ratio of bandwidth resources, Unloading similar data to the server for users, allocating bandwidth resources in proportion, The bandwidth allocation ratio for users to offload local models to the server. K is the total number of communication rounds;

[0177] The constraints of the model are:

[0178] F(w)≤ζ

[0179] KT≤T max

[0180]

[0181]

[0182]

[0183]

[0184] Where, is the maximum allowed learning loss, T max is the maximum allowed delay for the entire federation training, Is a positive integer.

[0185] Based on the characteristics of a collaborative hybrid federated learning architecture, this paper designs a three-stage federated training process: sensor-user-server. The process calculates the energy and time consumed in each stage and establishes a mathematical model that minimizes energy consumption within the constraints of model loss and latency. The paper then analyzes the optimization variables and the characteristics of the mathematical model to determine the optimal variable values.

[0186] In summary, the present invention has the following advantages and beneficial effects:

[0187] 1. Taking into account the heterogeneous characteristics of network device resources and data, this paper designs a hierarchical collaborative hybrid federated learning framework, and for the first time proposes a scenario in which each user plays the dual role of relay and server computing at the same time.

[0188] 2. Based on the proposed collaborative hybrid federated learning framework, this paper proposes an energy consumption minimization problem under the constraints of learning loss and delay by optimizing bandwidth and number of communication rounds, effectively balancing the communication cost and the accuracy of the federated learning model.

[0189] 3. The present invention is simple and effective, and the performance of the proposed framework is higher than that of traditional federated learning. The present invention only requires a user-server interaction model, and the total amount of interaction is small.

[0190] The present invention also discloses an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the resource allocation optimization method for collaborative hybrid federated learning described above are implemented. The electronic device of the present invention can execute the resource allocation optimization method for collaborative hybrid federated learning of the present invention, and can execute any combination of the steps of the method embodiments, thereby possessing the corresponding functions and beneficial effects of the method.

[0191] The present invention also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the aforementioned methods for optimizing resource allocation for collaborative hybrid federated learning. The computer-readable storage medium of the present invention can execute the method for optimizing resource allocation for collaborative hybrid federated learning of the present invention, and can execute any combination of the steps of the method embodiments, thereby achieving the corresponding functions and beneficial effects of the method.

[0192] Although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It will also be understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the system disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art will be able to implement the present invention set forth in the claims using ordinary techniques without undue experimentation. It will also be understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0193] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0194] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0195] The various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0196] The above description is a detailed description of the preferred embodiments of the present invention, but the embodiments are not intended to limit the scope of the patent application of the present invention. Any equivalent changes or modifications made under the technical spirit disclosed by the present invention should fall within the patent scope covered by the present invention.

Claims

1. A resource allocation optimization method for collaborative hybrid federated learning, characterized in that: The following steps are involved: S1. Establish a collaborative hybrid federated learning system for devices, users, and servers to calculate the data similarity between devices and users and determine the distribution of data between users and servers. S2. Calculate the energy consumption and latency of the devices, users, and servers at each stage of federated training in the collaborative hybrid federated learning system to obtain the system's energy consumption and latency. S3. Determine the bandwidth resource ratio between devices and users, as well as the number of communication iterations between users and servers, establish a model with the goal of minimizing energy consumption, and determine the constraints of the model; S4. Decouple the model optimization problem into a bandwidth resource allocation optimization subproblem and a communication round number optimization subproblem, solve the subproblems, and obtain a resource allocation optimization solution; The collaborative hybrid federated learning system performs federated training, which includes the following stages: Phase 1: Each device offloads data to the corresponding service user; Phase 2: After each user receives the device data for the corresponding service, it first determines whether the device data is similar to the user's data. If the device data is similar, the device data is directly forwarded to the server; If the device data is not similar to the user's, the device data is added to the user's local data for local training to obtain a local model, and then the user transmits the local model to the server; Phase 3: The server uses the user's transmitted data to perform local training to obtain a local model. The server aggregates the trained local model and the local model sent by the user into a global model. Calculate the energy consumption and latency of the device by: In the first phase, the achievable data rate for the device to offload data to the user is: R i′ =b i′ C i′ Where, β i′ is the bandwidth ratio allocated to device i′, C i′ is the channel capacity of device i′, R i′ The achievable data rate for device i′ to offload data to user i; The channel capacity calculation formula of the device is: Where B is the total bandwidth, p i′ is the transmission power of device i′, h i′,i is the channel power gain from device i′ to user i, n0 is the background noise power; The time consumed by the device to transmit data is: Where D i′ is the local data of device i′, The time for device i′ to transmit data; The energy consumed by the device to transmit data is: Where, Energy consumption of transmitting data to device i′; Calculate the user's energy consumption and latency using the following method: In the second stage, users offload similar data to the server, and the achievable data rate for transmitting data is: Where, The bandwidth ratio allocated to user i when offloading similar data to the server; C i is the channel capacity of user i; The achievable data rate for offloading similar data to the server for user i; The calculation formula for the user's channel capacity is: Where p i is the transmission power of user i, h i is the channel power gain from user i to the server; The user offloads similar data to the server, and the time consumed for data transmission is: Where, Represents the device set served by user i; represents a set of devices that are dissimilar to the data of user i, The user offloads similar data to the server, and the energy consumed in transmitting data is: Where, The energy consumed in transmitting data for user i; In the second stage, the user offloads the local model to the server, and the achievable data rate of the transmitted data is: in, The bandwidth ratio allocated to user i when offloading the local model to the server; The achievable data rate for offloading the local model to the server for user i; The user unloads the local model to the server, and the time consumed for data transmission is: Where W is the size of the local model in bits; The time consumed in transmitting the local model for user i; The user offloads the local model to the server, and the energy consumed by transmitting data is: Where, Energy consumed by transmitting the local model for user i; For user local training, the time consumed by each user is: Where D i is the local data of user i, c is the number of CPU cycles in model training, The time consumed by user i for local training, f i is the CPU cycle speed of user i; The energy consumed by each user is: Where ξ is the chip capacitance related to the CPU, Energy consumed for local training of user i; Calculate the server's energy consumption and latency using: In the third stage, the time consumed by the server training model is: Where, represents the user set; f I+1 is the CPU cycle speed of the server; The time it takes to train the model for the server; The energy consumed by the server training model is: Where, e I+1 Energy consumed by the server to train the model; The total delay of each round of communication in the system is: T=max{T1,T2} Where T1 is the time for data transmission from device to user, from user to server, and for server calculation; T2 is the time for data transmission from device to user, from user calculation, and from user to server local model; and T is the total delay of each round of communication in the system. The calculation formulas for T1 and T2 are: Where γ is the number of local training iterations in each round of communication; The total energy consumed by the system in the communication link is obtained as: Where E is the total energy consumed by the system in the communication link; The model established with the goal of minimizing energy consumption is: Where β is the allocation ratio of bandwidth resources, Unloading similar data to the server for users, allocating bandwidth resources in proportion, The bandwidth allocation ratio for users to offload local models to the server. K is the total number of communication rounds; The constraints of the model are: F(w)≤ζ KT≤T max Where ζ is the maximum allowed learning loss, T max is the maximum allowed delay for the entire federation training, Is a positive integer.

2. The resource allocation optimization method for collaborative hybrid federated learning according to claim 1 is characterized in that: Calculate the data similarity between the device and the user, and determine the distribution of data between the user and the server. Specifically: The cosine similarity is used to calculate the data similarity between the device and the user. A similarity threshold is set. If the data similarity between the device and the user is higher than the similarity threshold, the data of the device and the user are considered similar. Otherwise, the data of the device and the user are considered dissimilar. If the data of the device and the user are similar, the user relays the device data to the server for training. If the data of the device and the user are not similar, the user uses the device data for user local training.

3. A collaborative hybrid federated learning system, applied to the collaborative hybrid federated learning resource allocation optimization method according to any one of claims 1 to 2, characterized in that: include: One server, multiple users, and multiple devices. Multiple users and one server form the edge layer, and multiple devices form the perception layer. Each user serves a group of disconnected devices, and each user and each device stores local data. Each user is used to obtain data from the corresponding service device, determine the data similarity between the device and the user, add the device data to the local data for user local training, or send the device data to the server for training through the user relay.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the resource allocation optimization method for collaborative hybrid federated learning as described in any one of claims 1 to 2 are implemented.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the resource allocation optimization method for collaborative hybrid federated learning as described in any one of claims 1 to 2 are implemented.

Images