Collaborative Federated Learning Optimization Method and Related Equipment Based on Graph Neural Networks

Abstract

This application provides a collaborative federated learning optimization method and related device based on graph neural networks. The method includes: determining a position information matrix and an adjacency matrix between a target node and its neighboring nodes; inputting the position information matrix and the adjacency matrix into a pre-constructed graph neural network model to obtain an embedding vector of the target node; calculating a connection probability distribution between the target node and its neighboring nodes based on the embedding vector; determining a topological link between the target node and its neighboring nodes based on the connection probability distribution; and performing collaborative federated learning model aggregation through the topological link.

Description

Technical Field

[0001] This application relates to the fields of communication and federated learning technologies, and in particular to a collaborative federated learning optimization method and related equipment based on graph neural networks. Background Technology

[0002] Federated learning (FL) is a framework that enables devices to collaboratively train machine learning (ML) models without exchanging data, effectively protecting data privacy. However, standard FL requires devices to transmit their FL models to a parameter server, which can limit its application due to the limited wireless resources (i.e., energy and bandwidth) in IoT networks. Collaborative Federated Learning (CFL) solves this problem. CFL allows devices to collaboratively train ML models by exchanging their local FL model parameters with neighboring devices, without relying on a parameter server. Compared to standard FL, CFL enables more devices to join FL training. For example, due to limited wireless resources, parameter servers can connect to nearby devices instead of directly connecting to them, thus participating in the CFL framework.

[0003] However, implementing CFL over wireless networks presents several challenges, including optimizing CFL learning efficiency, device connectivity, and distributed gradient updates. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose a collaborative federated learning optimization method and related equipment based on graph neural networks to solve or partially solve the above problems.

[0005] To achieve the above objectives, this application provides a collaborative federated learning optimization method based on graph neural networks, comprising:

[0006] Determine the position information matrix and adjacency matrix between the target node and its neighboring nodes;

[0007] The location information matrix and the adjacency matrix are input into a pre-constructed graph neural network model to obtain the embedding vector of the target node;

[0008] Calculate the connection probability distribution between the target node and its neighboring nodes based on the embedding vector;

[0009] The topological links between the target node and its neighboring nodes are determined based on the connection probability distribution, and the collaborative federated learning model is aggregated through the topological links.

[0010] In one possible implementation, determining the position information matrix and adjacency matrix between the target node and its neighboring nodes includes:

[0011] Determine the distance between the target node and several mobile nodes within a preset area;

[0012] The neighboring nodes of the target node are determined from the plurality of mobile nodes based on the distance;

[0013] Obtain the first location information of the target node and the second location information of the neighboring nodes;

[0014] The location information matrix and the adjacency matrix are determined based on the first location information and the second location information.

[0015] In one possible implementation, the neighboring nodes include: a first neighboring node and a second neighboring node; the distance between the first neighboring node and the target node is less than the distance between the second neighboring node and the target node;

[0016] The step of inputting the location information matrix and the adjacency matrix into a pre-constructed graph neural network model to obtain the embedding vector of the target node includes:

[0017] The location information matrix and the adjacency matrix are input into a pre-constructed graph neural network model, and the target node and the first neighboring node are aggregated to obtain the first aggregation matrix output by the first hidden layer of the graph neural network.

[0018] Based on the first hidden layer matrix, the target node and the second neighboring node are aggregated to obtain the second aggregation matrix output by the second hidden layer of the graph neural network;

[0019] The embedding vector is obtained based on the second aggregation matrix.

[0020] In one possible implementation, calculating the connection probability distribution between the target node and its neighboring nodes based on the embedding vector includes:

[0021] Calculate the inner product of the embedding vector of the target node and the embedding vectors of the neighboring nodes;

[0022] The inner product is normalized using an activation function to obtain the connection probability distribution between the target node and its neighboring nodes.

[0023] In one possible implementation, determining the topology links between the target node and the neighboring nodes based on the connection probability distribution includes:

[0024] Determine whether the target node and the neighboring nodes are each other's nearest neighbors; wherein, being each other's nearest neighbors indicates that the target node has the highest probability of connecting to the neighboring nodes; and the neighboring nodes have the highest probability of connecting to the target node.

[0025] In response to the fact that the target node and the neighboring node are each other's nearest neighbors, it is determined that there is a topological link between the target node and the neighboring node.

[0026] In one possible implementation, the method further includes:

[0027] In response to the fact that the target node and the neighboring node are not each other's nearest neighbors, it is determined whether the target node and the neighboring node are each other's second nearest neighbors; wherein, being each other's second nearest neighbors indicates that the connection probability of the target node to the neighboring node is less than the connection probability to the nearest neighbor, and the connection probability of the target node to the nearest neighbor is less than the connection probability of the nearest neighbor to the target node.

[0028] In response to the fact that the target node and the neighboring node are each other's next-nearest nodes, it is determined that there is a topological link between the target node and the neighboring node.

[0029] In one possible implementation, the collaborative federated learning model aggregation via the topology links includes:

[0030] In response to determining that a topology link exists between the target node and the neighboring node;

[0031] The first local model of the target node is updated based on the second local model sent to the target node by the neighboring nodes, thereby realizing the collaborative federated learning model aggregation between the target node and the neighboring nodes.

[0032] For the same purpose, this application also proposes a cooperative federated learning optimization device based on graph neural networks, comprising:

[0033] The location determination module is configured to determine the location information matrix and adjacency matrix between the target node and its neighboring nodes;

[0034] The embedding module is configured to input the location information matrix and the adjacency matrix into a pre-constructed graph neural network model to obtain the embedding vector of the target node;

[0035] The probability determination module is configured to calculate the connection probability distribution between the target node and the neighboring nodes based on the embedding vector;

[0036] The collaboration module is configured to determine the topological links between the target node and the neighboring nodes based on the connection probability distribution, and to perform collaborative federated learning model aggregation through the topological links.

[0037] For the purposes described above, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any of the above descriptions.

[0038] For the purposes described above, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to perform any of the methods described above.

[0039] As described above, the collaborative federated learning optimization method and related device based on graph neural networks provided in this application first determine the position information matrix and adjacency matrix between the target node and its neighboring nodes. Then, the position information matrix and adjacency matrix are input into a pre-constructed graph neural network model to obtain the embedding vector of the target node. Further, the connection probability distribution between the target node and its neighboring nodes is calculated based on the embedding vector. Finally, the topological links between the target node and its neighboring nodes are determined based on the connection probability distribution, and the collaborative federated learning model is aggregated through these topological links. This application enables each mobile device within the target area to independently determine the transmission path of its federated learning parameters using the position and connection information of neighboring devices, thus deriving the optimal FL parameter transmission path. Furthermore, the aggregation of the collaborative federated learning model is achieved through the optimal FL parameter transmission path, optimizing the data transmission path, improving FL parameter transmission efficiency, and significantly reducing CFL training energy consumption. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram illustrating an application scenario of the collaborative federated learning optimization method based on graph neural networks provided in this application embodiment.

[0042] Figure 2 This is a flowchart illustrating the collaborative federated learning optimization method based on graph neural networks provided in an embodiment of this application.

[0043] Figure 3This is a schematic diagram of a collaborative federated learning optimization device based on graph neural networks provided in an embodiment of this application.

[0044] Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0046] It should be noted that, unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0047] As described in the background section, existing CFL algorithms are all based on static network topology design, meaning the connections between devices do not change over time. Therefore, these studies do not consider the impact of network topology on CFL training, leading to redundancy in CFL parameter transmission and increased communication overhead. Furthermore, these studies also fail to consider the impact of network dynamics (such as device movement patterns) on CFL training, which will limit the application of CFL algorithms.

[0048] In addition, during the development of this application, the applicant discovered that existing technologies utilize centralized controllers to schedule device connections. However, in practical IoT systems, direct connection to a central controller may face challenges related to resource constraints such as energy and bandwidth. For example, large-scale sensor networks may lack a fusion center, and sensors may only communicate with other nearby sensors. These studies are limited to resource allocation using GNNs in fixed D2D networks, which limits their adaptability in scenarios requiring dynamic pairing. Therefore, designing a novel CFL deployment method that considers both device connectivity and deployment in mobile systems is an urgent problem to be solved.

[0049] Therefore, this application proposes a collaborative federated learning optimization method and related equipment based on graph neural networks.

[0050] The technical solution of this application will be further described in detail below through specific embodiments.

[0051] refer to Figure 1 This is a schematic diagram illustrating an application scenario of the collaborative federated learning optimization method based on graph neural networks provided in this application embodiment.

[0052] This application is primarily applied to distributed machine learning scenarios in mobile user environments. In a fully distributed scenario, each user, based on their own location and the location information of their neighbors, selects some neighbors to exchange local models, thus meeting the system's requirements for energy consumption, latency, and federated learning efficiency. By optimizing transmission energy, federated learning performance can be guaranteed, resolving the trade-off between transmission resource consumption and learning efficiency in distributed machine learning scenarios.

[0053] like Figure 1 As shown, multiple mobile devices move freely in space and send their location information to their K nearest neighbor devices. This space can be a pre-defined area. After collecting the location information of neighboring devices, each device needs to calculate its connection probability with its neighboring nodes using a graph neural network and then send this probability to its K nearest neighbor devices. Each device considers both its own probability of connecting to neighboring devices and the probability of a neighbor connecting to it, ultimately determining a topology for exchanging local models. After obtaining the topology, the transmission power needs to be determined based on latency and energy limitations to ensure that all model parameters are transmitted within a fixed time. Based on this, each user, according to its own location and the locations of neighboring devices, determines a topology for model exchange and fusion in a completely distributed manner.

[0054] refer to Figure 2 This is a flowchart illustrating the collaborative federated learning optimization method based on graph neural networks provided in this application embodiment.

[0055] Step S201: Determine the position information matrix and adjacency matrix between the target node and its neighboring nodes.

[0056] In practical implementation, within a distributed machine learning scenario for mobile users, a mobile node is randomly selected as the target node. Neighboring nodes are then selected from the target node's surrounding devices according to preset conditions. Each mobile node can send its own GPS location information to other mobile nodes, and each node can determine its positional relationship with other nodes based on its own GPS location information and that of other nodes. All mobile nodes can send their GPS location information to surrounding nodes based on past experience or through broadcasting.

[0057] As an optional embodiment, the distance between the target node and several mobile nodes within a preset area can first be determined. Based on the distance, neighboring nodes of the target node are determined from among the mobile nodes. Specifically, if the distance is within a preset range, the mobile node corresponding to that distance is determined to be a neighboring node of the target node; if the distance is not within the preset range, the mobile node corresponding to that distance is determined not to be a neighboring node of the target node. Further, first location information of the target node and second location information of the neighboring nodes are obtained. The location information is the GPS location information of each node. A location information matrix and an adjacency matrix are determined based on the first and second location information.

[0058] As an optional embodiment, the location information matrix and adjacency matrix refer to the matrix relationships between the target node and its neighboring nodes. Let L 1 (i,k) represents the set of the first neighboring nodes that can be directly connected to the target node, L 2 (i,k) represents the set of the second nearest neighbors of the target node, that is, the set of the "nearest neighbors" of the target node's nearest neighbor.

[0059] As an optional embodiment, after determining its own and its neighboring nodes' positions, the target node can determine an adjacency matrix based on these positions. The target node can then construct a position information matrix between itself and its neighboring nodes based on the collected position information of its neighboring nodes and the adjacency matrix. And the adjacency matrix R(i,k).

[0060] Step S202: Input the location information matrix and the adjacency matrix into the pre-constructed graph neural network model to obtain the embedding vector of the target node.

[0061] Graph Neural Networks (GNNs) are a collective term for algorithms that use neural networks to learn graph-structured data, extract and discover features and patterns within it, and fulfill graph learning tasks such as clustering, classification, prediction, segmentation, and generation. GNNs leverage the capabilities of neural networks, such as deep feature extraction, to process graph-structured data. Examples include social network graphs, traffic route maps, relationship graphs, molecular structure graphs, and computational network topology graphs. These data can all serve as input to a GNN. After processing through specific neural network architectures, such as MLPs, CNNs, and RNNs, based on graph structures, functions such as classifying graph representations and predicting nodes or edges can be achieved.

[0062] As an optional embodiment, the neighboring nodes include a first neighboring node and a second neighboring node. The distance between the first neighboring node and the target node is less than the distance between the second neighboring node and the target node. The position information matrix and the adjacency matrix are input into a pre-built graph neural network model. The target node and the first neighboring node are aggregated to obtain a first aggregation matrix output by the first hidden layer of the graph neural network. Further, the target node and the second neighboring node are aggregated according to the first hidden layer matrix to obtain a second aggregation matrix output by the second hidden layer of the graph neural network. Finally, the embedding vector is obtained according to the second aggregation matrix.

[0063] As an optional embodiment, the hidden layers of the graph neural network model pre-built in this application consist of two graph convolutional network layers (GCNs), represented as follows:

[0064]

[0065]

[0066] in, R(i, k) is the adjacency matrix, and I = diag(1) is a diagonal matrix with diagonal elements of 1. r i Φ represents the element in row i of the adjacency matrix R(i, k). i For location information matrix, and These are the parameters of the GNN model. To correct the linear unit activation function, V1 and V2 are the output dimensions. This is the output of the first GCN layer. This is the output of the second GCN layer.

[0067] Step S203: Calculate the connection probability distribution between the target node and the neighboring nodes based on the embedding vector.

[0068] As an optional embodiment, it is necessary to calculate the inner product of the embedding vector of the target node and the embedding vector of the neighboring nodes, and then use an activation function to normalize the inner product to obtain the connection probability distribution between the target node and the neighboring nodes.

[0069] In practical implementation, the connection probability distribution can be expressed by the following formula:

[0070]

[0071] in, z represents the probability that the target node is connected to its neighboring nodes. i This represents the i-th row of the Z matrix. It should be noted that the prediction result μ... iIt is not only determined by the transfer matrix of the FL model. Since the data labels used to train the GNN model are generated from the optimal U and P, the transfer matrix U of the FL model and the optimal transmit power vector P can both be used to train the GNN model.

[0072] As an optional embodiment, given the predicted connection probability μ i Further determination of the FL model transfer matrix u of the target node is needed. i First, the target node transmits its predicted connection probability to its neighboring nodes. Then, each target node receives the FL model transfer matrix to determine μ. i First, we can define μ i A set of elements in the middle. n represents the maximum connection probability between the target node and its neighboring nodes. i，max This is the index of the neighboring node with the highest connection probability. Further, we can define... Indicates the target node and excluding n. i，max The connection probability of neighboring nodes other than the target node, i.e., the neighboring device with the second highest connection probability to the target node.

[0073] In the FL model, u i，j The connection probability distribution condition can be expressed as:

[0074]

[0075] Where, n i，max It is the neighboring node with the highest probability of being connected to the target node i, n′ i，max It is the index of the node with the second highest probability of being connected to the target node i. The target node i and its neighboring nodes n i，max The connection probability, It is the neighboring node n i，max The connection probability with target node i It is the target node with the highest probability of being connected to its neighboring nodes.

[0076] Step S204: Determine the topological links between the target node and the neighboring nodes based on the connection probability distribution, and perform collaborative federated learning model aggregation through the topological links.

[0077] As an optional embodiment, there are connection probabilities of the target node to neighboring nodes and connection probabilities of neighboring nodes to the target node.

[0078] Specifically, it is necessary to determine whether the target node and its neighboring nodes are each other's nearest neighbors. Being each other's nearest neighbors indicates that the target node has the highest probability of connecting to its neighboring nodes, and the neighboring nodes also have the highest probability of connecting to the target node. This is the first criterion for determining whether a connection probability distribution condition is met. In response to the target node and its neighboring nodes being each other's nearest neighbors, it is determined that a topological link exists between the target node and its neighboring nodes.

[0079] As an optional embodiment, if the target node and its neighboring nodes are not each other's nearest neighbors, it is necessary to determine whether the target node and its neighboring nodes are each other's second nearest neighbors. Here, being a second nearest neighbor indicates that the probability of the target node connecting to a neighboring node is less than the probability of connecting to its nearest neighbor, and the probability of the target node connecting to its nearest neighbor is less than the probability of the nearest neighbor connecting to the target node. For example, if node i and node z are devices with the highest probability of connection, then the probability of node i connecting to z is numerically less than the probability of node z connecting to node i. Therefore, node i needs to find the node with the second highest probability of connection. The node connected to node i in this case is the second nearest neighbor.

[0080] Furthermore, in response to the fact that the target node and its neighboring nodes are each other's next-nearest nodes, it is determined that there is a topological link between the target node and its neighboring nodes.

[0081] It should be noted that the nearest neighbor and the second nearest neighbor are both determined from the first nearest neighbor of the target node. That is, the best among the first nearest neighbors is selected. Both the nearest neighbor and the second nearest neighbor can directly connect to the target node, but they differ in the connection probability.

[0082] Once the connection between the target node and its neighboring nodes is determined, the topology links between the target node and its neighboring nodes and the connection matrix generated for the topology links can be determined.

[0083] As an optional implementation, both the target node and its neighboring nodes need to train a collaborative federated learning model based on their own data. The goal of the training is to minimize the global loss function of all data samples, expressed as:

[0084]

[0085] Furthermore, the back propagation (BP) algorithm based on stochastic gradient descent (SGD) is used to update the local FL model parameters of each node.

[0086] Specifically, including:

[0087] S1: Each node randomly initializes its local model;

[0088] S2: Each node updates its local model using the data samples it has collected;

[0089] S3: Each node exchanges the updated local model with its own selected subset of devices;

[0090] S4: Each node will aggregate the FL models received from other nodes and generate its own FL model.

[0091] Repeat steps S1-S4 until the optimal vector g is found.

[0092] Based on the previously determined topology and connection matrix, the target node sends its trained local model to its corresponding neighboring nodes and receives local models sent by the neighboring nodes. The collaborative federated learning model is then aggregated according to the following formula:

[0093]

[0094] Among them, g′ i，t+1 λ is the local model updated in iteration t (i.e., the updated local model), where λ is the learning rate. y is the subset of devices selected by the node from the training data sample set during iteration t, where x is the data input for machine learning and y is the label corresponding to the data.

[0095] As described above, the collaborative federated learning optimization method based on graph neural networks provided in this application first determines the position information matrix and adjacency matrix between the target node and its neighboring nodes. Then, the position information matrix and adjacency matrix are input into a pre-constructed graph neural network model to obtain the embedding vector of the target node. Further, the connection probability distribution between the target node and its neighboring nodes is calculated based on the embedding vector. Finally, the topological links between the target node and its neighboring nodes are determined based on the connection probability distribution, and the collaborative federated learning model is aggregated through these topological links. This application enables each mobile device within the target area to independently determine the transmission path of its federated learning parameters using the position and connection information of neighboring devices, thus deriving the optimal FL parameter transmission path. Furthermore, the aggregation of the collaborative federated learning model is achieved through the optimal FL parameter transmission path, optimizing the data transmission path, improving FL parameter transmission efficiency, and significantly reducing CFL training energy consumption.

[0096] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0097] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0098] Based on the same inventive concept, corresponding to the cooperative federated learning optimization method based on graph neural networks provided in any of the above embodiments, this application also provides a cooperative federated learning optimization device based on graph neural networks.

[0099] refer to Figure 3 This is a schematic diagram of a collaborative federated learning optimization device based on graph neural networks provided in an embodiment of this application.

[0100] The device includes:

[0101] The location determination module 301 is configured to determine the location information matrix and adjacency matrix between the target node and its neighboring nodes;

[0102] Embedding module 302 is configured to input the location information matrix and the adjacency matrix into a pre-constructed graph neural network model to obtain the embedding vector of the target node;

[0103] The probability determination module 303 is configured to calculate the connection probability distribution between the target node and the neighboring nodes based on the embedding vector;

[0104] The collaboration module 304 is configured to determine the topological links between the target node and the neighboring nodes based on the connection probability distribution, and to perform collaborative federated learning model aggregation through the topological links.

[0105] Optionally, the position determination module 301 is further configured to:

[0106] Determine the distance between the target node and several mobile nodes within a preset area;

[0107] The neighboring nodes of the target node are determined from the plurality of mobile nodes based on the distance;

[0108] Obtain the first location information of the target node and the second location information of the neighboring nodes;

[0109] The location information matrix and the adjacency matrix are determined based on the first location information and the second location information.

[0110] Optionally, the neighboring nodes include: a first neighboring node and a second neighboring node; the distance between the first neighboring node and the target node is less than the distance between the second neighboring node and the target node;

[0111] The location embedding module 302 is further configured to:

[0112] The location information matrix and the adjacency matrix are input into a pre-constructed graph neural network model, and the target node and the first neighboring node are aggregated to obtain the first aggregation matrix output by the first hidden layer of the graph neural network.

[0113] Based on the first hidden layer matrix, the target node and the second neighboring node are aggregated to obtain the second aggregation matrix output by the second hidden layer of the graph neural network;

[0114] The embedding vector is obtained based on the second aggregation matrix.

[0115] Optionally, the probability determination module 303 is further configured to:

[0116] Calculate the inner product of the embedding vector of the target node and the embedding vectors of the neighboring nodes;

[0117] The inner product is normalized using an activation function to obtain the connection probability distribution between the target node and its neighboring nodes.

[0118] Optionally, the collaboration module 304 is further configured to:

[0119] Determine whether the target node and the neighboring nodes are each other's nearest neighbors; wherein, being each other's nearest neighbors indicates that the target node has the highest probability of connecting to the neighboring nodes; and the neighboring nodes have the highest probability of connecting to the target node.

[0120] In response to the fact that the target node and the neighboring node are each other's nearest neighbors, it is determined that there is a topological link between the target node and the neighboring node.

[0121] Optionally, the collaboration module 304 is further configured to:

[0122] In response to the fact that the target node and the neighboring node are not each other's nearest neighbors, it is determined whether the target node and the neighboring node are each other's second nearest neighbors; wherein, being each other's second nearest neighbors indicates that the connection probability of the target node to the neighboring node is less than the connection probability to the nearest neighbor, and the connection probability of the target node to the nearest neighbor is less than the connection probability of the nearest neighbor to the target node.

[0123] In response to the fact that the target node and the neighboring node are each other's next-nearest nodes, it is determined that there is a topological link between the target node and the neighboring node.

[0124] Optionally, the collaboration module 304 is further configured to:

[0125] In response to determining that a topology link exists between the target node and the neighboring node;

[0126] The first local model of the target node is updated based on the second local model sent to the target node by the neighboring nodes, thereby realizing the collaborative federated learning model aggregation between the target node and the neighboring nodes.

[0127] For ease of description, the above system is described by dividing it into various modules based on their functions. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0128] The system described in the above embodiments is used to implement the corresponding collaborative federated learning optimization method based on graph neural networks in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0129] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the cooperative federated learning optimization method based on graph neural networks as described in any of the above embodiments.

[0130] Figure 4 This illustration shows a more specific hardware structure diagram of an electronic device provided in this embodiment. The device may include: a processor 410, a memory 420, an input / output interface 430, a communication interface 440, and a bus 450. The processor 410, memory 420, input / output interface 430, and communication interface 440 are interconnected internally via the bus 450.

[0131] The processor 410 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0132] The memory 420 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 420 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 420 and is called and executed by the processor 410.

[0133] Input / output interface 430 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0134] The communication interface 440 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (e.g., USB, Ethernet cable) or wireless means (e.g., mobile network, Wi-Fi, Bluetooth).

[0135] Bus 450 includes a pathway for transmitting information between various components of the device (e.g., processor 410, memory 420, input / output interface 430, and communication interface 440).

[0136] It should be noted that although the above-described device only shows the processor 410, memory 420, input / output interface 430, communication interface 440, and bus 450, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0137] The electronic devices described above are used to implement the corresponding collaborative federated learning optimization methods based on graph neural networks in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0138] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the cooperative federated learning optimization method based on graph neural networks as described in any of the above embodiments.

[0139] The aforementioned non-transitory computer-readable storage media can be any available medium or data storage device that a computer can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0140] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the cooperative federated learning optimization method based on graph neural networks as described in any of the embodiments in the exemplary method section above, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0141] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0142] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0143] While the spirit and principles of this application have been described with reference to several specific embodiments, it should be understood that this application is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for convenience of expression. This application is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the appended claims is to be interpreted in the broadest sense, thereby encompassing all such modifications and equivalent structures and functions.

Claims

1. A collaborative federated learning optimization method based on graph neural networks, characterized in that, include: Determining the location information matrix and adjacency matrix between a target node and its neighboring nodes includes: determining the distance between the target node and a plurality of mobile nodes within a preset area; determining the neighboring nodes of the target node from the plurality of mobile nodes based on the distance; acquiring first location information of the target node and second location information of the neighboring nodes; and determining the location information matrix and the adjacency matrix based on the first location information and the second location information. The target node is a randomly selected mobile node in a distributed machine learning scenario within a mobile user scenario; the neighboring nodes are selected from nearby devices around the target node according to preset conditions. The location information matrix and the adjacency matrix are input into a pre-constructed graph neural network model to obtain the embedding vector of the target node; Calculate the connection probability distribution between the target node and its neighboring nodes based on the embedding vector; The topological links between the target node and its neighboring nodes are determined based on the connection probability distribution, and the collaborative federated learning model is aggregated through the topological links.

2. The method according to claim 1, characterized in that, The neighboring nodes include: a first neighboring node and a second neighboring node; the distance between the first neighboring node and the target node is less than the distance between the second neighboring node and the target node; The step of inputting the location information matrix and the adjacency matrix into a pre-constructed graph neural network model to obtain the embedding vector of the target node includes: The location information matrix and the adjacency matrix are input into a pre-constructed graph neural network model, and the target node and the first neighboring node are aggregated to obtain the first aggregation matrix output by the first hidden layer of the graph neural network. The target node and the second neighboring node are aggregated according to the first aggregation matrix to obtain the second aggregation matrix output by the second hidden layer of the graph neural network; The embedding vector is obtained based on the second aggregation matrix.

3. The method according to claim 1, characterized in that, The step of calculating the connection probability distribution between the target node and its neighboring nodes based on the embedding vector includes: Calculate the inner product of the embedding vector of the target node and the embedding vectors of the neighboring nodes; The inner product is normalized using an activation function to obtain the connection probability distribution between the target node and its neighboring nodes.

4. The method according to claim 1, characterized in that, Determining the topology links between the target node and its neighboring nodes based on the connection probability distribution includes: Determine whether the target node and the neighboring nodes are each other's nearest neighbors; wherein, being each other's nearest neighbors indicates that the target node has the highest probability of connecting to the neighboring nodes; and the neighboring nodes have the highest probability of connecting to the target node. In response to the fact that the target node and the neighboring node are each other's nearest neighbors, it is determined that there is a topological link between the target node and the neighboring node.

5. The method according to claim 4, characterized in that, The method further includes: In response to the fact that the target node and the neighboring node are not each other's nearest neighbors, it is determined whether the target node and the neighboring node are each other's second nearest neighbors; wherein, being each other's second nearest neighbors indicates that the connection probability of the target node to the neighboring node is less than the connection probability to the nearest neighbor, and the connection probability of the target node to the nearest neighbor is less than the connection probability of the nearest neighbor to the target node. In response to the fact that the target node and the neighboring node are each other's next-nearest nodes, it is determined that there is a topological link between the target node and the neighboring node.

6. The method according to claim 4 or 5, characterized in that, The aggregation of collaborative federated learning models through the topology links includes: In response to determining that a topology link exists between the target node and the neighboring node; The first local model of the target node is updated based on the second local model sent to the target node by the neighboring nodes, thereby realizing the collaborative federated learning model aggregation between the target node and the neighboring nodes.

7. A cooperative federated learning optimization device based on graph neural networks, characterized in that, include: A location determination module is configured to determine a location information matrix and an adjacency matrix between a target node and its neighboring nodes, including: determining the distance between the target node and a plurality of mobile nodes within a preset area; determining neighboring nodes of the target node from the plurality of mobile nodes based on the distance; acquiring first location information of the target node and second location information of the neighboring nodes; and determining the location information matrix and the adjacency matrix based on the first location information and the second location information. The target node is a randomly selected mobile node in a distributed machine learning scenario within a mobile user scenario; the neighboring nodes are selected from nearby devices around the target node according to preset conditions. The embedding module is configured to input the location information matrix and the adjacency matrix into a pre-constructed graph neural network model to obtain the embedding vector of the target node; The probability determination module is configured to calculate the connection probability distribution between the target node and the neighboring nodes based on the embedding vector; The collaboration module is configured to determine the topological links between the target node and the neighboring nodes based on the connection probability distribution, and to perform collaborative federated learning model aggregation through the topological links.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing the computer to perform the method of any one of claims 1 to 6.

Images