Method and apparatus for model training

Abstract

The application provides a model training method and device. The method comprises: obtaining first capability information and second capability information of a client; determining, based on the first capability information and the second capability information, whether the client reports model information to a cloud server or an edge server; and sending first indication information to the client, the first indication information being used to instruct the client to report model information to the cloud server or the edge server. The model training method and device provided by the application can flexibly select a manner in which the client reports local model information according to the capability information of the client under a hybrid mode transverse federated training method, and can improve the efficiency of transverse federated training.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a method and apparatus for model training. Background Technology

[0002] In horizontal federated learning, a cloud-edge-client model exists. The process is as follows: First, each client reports its locally trained model information to its corresponding edge server. Then, all edge servers in the system aggregate the model information reported by all clients within their respective coverage areas to obtain an intermediate model, which is then reported to the cloud server. The cloud server further aggregates the intermediate models to obtain the final model, which is then distributed to all clients for local inference. However, this strict aggregation process via edge servers does not align with real-world network deployment scenarios. How to flexibly choose whether clients directly report model information to the cloud server or aggregate it through edge servers before reporting it is a problem that this application urgently needs to solve. Summary of the Invention

[0003] The model training method and apparatus provided in this application, under the hybrid mode of horizontal federated training, can flexibly select the method of client reporting model information according to the client's capability information, thereby improving the efficiency of horizontal federated training.

[0004] Firstly, a method for model training is provided. This method can be executed by a cloud server, an edge server, or a third-party server, or by a chip or circuit configured in the aforementioned servers; this application does not limit the execution of this method. The method includes: acquiring first capability information and second capability information of a client; determining a target server based on the first capability information and the second capability information, the target server being the server where the client reports model information, the target server including a cloud server or an edge server; and sending instruction information to the client, the instruction information instructing the client to report its model information to the target server.

[0005] According to the solution provided in this application, the server can determine the way the client reports model information based on the acquired client capability information, making the way the client reports model information in the system more flexible, thereby improving the efficiency of horizontal federated training.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, the first capability information includes at least one of the following information of the client: memory information, hard disk information, computing power information, load information, and channel bandwidth information; or the second capability information includes at least one of the following information of the client: client capability for federated learning, type of federated learning algorithm, and region information.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: obtaining the demand information of the cloud server, the demand information including the number of clients and / or data length; and determining the target server based on the first capability information, the second capability information, and the demand information.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the first instruction information may also include the address information of the cloud server and / or the address information of the edge server.

[0009] Based on the above solution, by obtaining the cloud server's demand information, the number of clients participating in training and / or the data length can be further controlled, thereby further improving training efficiency.

[0010] Secondly, a method for model training is provided. This method can be executed by a cloud server. Alternatively, it can be executed by a chip or circuit configured in the cloud server, without limitation in this application. The method includes: the cloud server receiving model information from a first client and model information from an edge server, wherein the model information from the edge server is determined based on the model information from a second client; the cloud server determining target model information based on the model information from the first client and the model information from the edge server; and the cloud server sending the target model information to the first client and the edge server.

[0011] It should be noted that in this application, the first client is different from the second client. It should be understood that during model training, model information from either client will not be reported to both the cloud server and the edge server simultaneously.

[0012] According to the solution provided in this application, the cloud server can train the target model by receiving model information from both the first client and the edge clients (e.g., by aggregating the model information from the first client and the edge clients to obtain the target model). This hybrid horizontal federated learning training system combines the advantages of a cloud-client horizontal federated learning training system, which allows for a large dataset, with the advantages of an edge-client horizontal federated learning training system, effectively alleviating network congestion caused by a large number of clients. Therefore, the training method of this system is more flexible and effective.

[0013] In conjunction with the second aspect, in some implementations of the second aspect, the method further includes: the cloud server sending a first instruction message to the first client, the first instruction message being used to instruct the first client to report the model information of the first client to the cloud server.

[0014] In conjunction with the second aspect, in some implementations of the second aspect, the first instruction information includes the address information of the cloud server.

[0015] In conjunction with the second aspect, in some implementations of the second aspect, the method further includes: the cloud server sending a second instruction message to the second client, the second instruction being used to instruct the second client to report the model information of the second client to the edge server.

[0016] In conjunction with the second aspect, in some implementations of the second aspect, the method further includes: the second indication information includes the address information of the edge server.

[0017] In conjunction with the second aspect, in some implementations of the second aspect, the method further includes: the cloud server sending a third instruction to the edge server, the third instruction being used to instruct the edge server to report the model information of the edge server to the cloud server.

[0018] In conjunction with the second aspect, in some implementations of the second aspect, the third instruction information includes the address information of the second client and / or the address information of the cloud server.

[0019] It should be understood that when the cloud server sends third instruction information to the edge server to instruct the edge server to report its model to the cloud server, this instruction information may include the address information of the second client. This address information is used by the edge server to determine that the client model information to be aggregated comes from the second client corresponding to the aforementioned address information. Furthermore, since the cloud server can communicate with multiple edge servers, the aforementioned address information of the cloud server may be the Internet Protocol (IP) address and / or channel port number used by each edge server to communicate with the cloud server.

[0020] In conjunction with the second aspect, in some implementations of the second aspect, the cloud server sends demand information to the edge server, the demand information including the sum of the number of the first client and the second client and / or the sum of the data lengths, the demand information being used to determine the first client and the second client.

[0021] Based on the above scheme, determining the method of client-reported model information through a cloud server can effectively improve the efficiency of horizontal federated training.

[0022] Thirdly, a method for model training is provided. This method can be executed by an edge server, or by a chip or circuit configured in the edge server, without limitation herein. The method includes: the edge server receiving model information from a second client; the edge server determining its own model information based on the model information from the second client; and the edge server sending the model information to the cloud server.

[0023] In conjunction with the third aspect, in some implementations of the third aspect, the method further includes: the edge server receiving first indication information from the cloud server, the first indication information being used to instruct the edge server to report the model information of the edge server to the cloud server.

[0024] In conjunction with the third aspect, in some implementations of the third aspect, the first instruction information includes the address information of the second client and / or the address information of the cloud server.

[0025] In conjunction with the third aspect, in some implementations of the third aspect, the method further includes: the edge server sending a second instruction message to the second client, the second instruction message being used to instruct the second client to report the model information of the second client to the edge server.

[0026] In conjunction with the third aspect, in some implementations of the third aspect, the second instruction information includes the address information of the edge server.

[0027] In conjunction with the third aspect, in some implementations of the third aspect, the method further includes: the edge server sending third indication information to the first client, the third indication information being used to instruct the first client to report the model information of the first client to the cloud server.

[0028] In conjunction with the third aspect, in some implementations of the third aspect, the second instruction information includes the address information of the cloud server.

[0029] Based on the above scheme, determining the method of client-reported model information through edge servers can improve the efficiency of horizontal federated learning training.

[0030] Fourthly, a model training device is provided, including: a processing module and a storage module.

[0031] Each module in the device is used to execute the steps of the method provided by the first aspect and any possible implementation of the first aspect.

[0032] In one design, the device is a model training chip, which may include input circuitry or interface for transmitting information or data, and output circuitry or interface for receiving information or data.

[0033] In another design, the device is a server, which can be a cloud server, an edge server, or another third-party server, and this application is not limited thereto. The server may include a transmitter for sending information or data, and a receiver for receiving information or data.

[0034] Fifthly, a model training device is provided, comprising: a processing unit and a storage unit.

[0035] Each unit in the device is used to perform each step of the method provided by the second aspect or any possible implementation of the second aspect.

[0036] In one design, the device is a model training chip, which may include input circuitry or interface for transmitting information or data, and output circuitry or interface for receiving information or data.

[0037] In another design, the device is a cloud server, which may include a transmitter for sending information or data and a receiver for receiving information or data.

[0038] Sixthly, a model training device is provided, including: a processing unit and a storage unit.

[0039] Each unit in the device is used to perform each step of the method provided by the third aspect or any possible implementation of the third aspect.

[0040] In one design, the device is a model training chip, which may include input circuitry or interface for transmitting information or data, and output circuitry or interface for receiving information or data.

[0041] In another design, the device is an edge server, which may include a transmitter for sending information or data and a receiver for receiving information or data.

[0042] Seventhly, a model training system is provided, including the model training equipment provided in the above aspects.

[0043] In one possible design, the model training system may also include other devices that interact with the model training device as provided in the embodiments of this application.

[0044] Eighthly, a computer program product is provided, comprising: a computer program (also referred to as code or instructions) that, when run, causes a computer to perform the methods described in the first to third aspects and any possible implementation thereof.

[0045] Ninth aspect, a computer-readable medium is provided that stores a computer program (also referred to as code or instructions) that, when executed on a computer, causes the computer to perform the methods of the first to third aspects and any possible implementation thereof.

[0046] In a tenth aspect, a chip system is provided, including a memory and a processor, the memory for storing a computer program and the processor for calling and running the computer program from the memory, such that a model training device equipped with the chip system performs the methods of the first to third aspects and any possible implementation thereof.

[0047] The chip system may include input circuits or interfaces for transmitting information or data, and output circuits or interfaces for receiving information or data. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of a horizontal federated learning approach.

[0049] Figure 2 This is a schematic diagram of a vertical federated learning model.

[0050] Figure 3 This is a schematic diagram of the training process for a horizontal federated learning model.

[0051] Figure 4 This is a schematic diagram of a horizontal federated learning architecture.

[0052] Figure 5 This is a schematic diagram of another horizontal federated learning architecture.

[0053] Figure 6 This is a schematic diagram of another horizontal federated learning architecture.

[0054] Figure 7 This is a schematic diagram of horizontal federated learning under a system architecture with a hybrid horizontal federated training mode.

[0055] Figure 8 This is a schematic diagram of horizontal federated learning under the 5G network architecture based on NWDAF.

[0056] Figure 9 This is a schematic diagram of horizontal federated learning in the NWDAF-RAN scenario.

[0057] Figure 10 This is a schematic flowchart illustrating a federated learning training method provided in an embodiment of this application.

[0058] Figure 11 This is a schematic flowchart of another federated learning training method provided in the embodiments of this application.

[0059] Figure 12 This is a schematic flowchart of another federated learning training method provided in the embodiments of this application.

[0060] Figure 13 This is a schematic flowchart of another federated learning training method provided in the embodiments of this application.

[0061] Figure 14 This is a schematic block diagram of a server in this application.

[0062] Figure 15 This is a schematic block diagram of a cloud server as described in this application.

[0063] Figure 16 This is a schematic block diagram of an example edge server in this application.

[0064] Figure 17 This is a schematic diagram of the model training device provided in the embodiments of this application.

[0065] Figure 18 This is a schematic diagram of another model training device provided in the embodiments of this application. Detailed Implementation

[0066] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0067] The wireless communication systems mentioned in the embodiments of this application include, but are not limited to: Global System for Mobile Communication (GSM) system, Long Term Evolution (LTE) Frequency Division Duplex (FDD) system, LTE Time Division Duplex (TDD) system, LTE system, LTE-Advanced (LTE-A) system, next-generation communication system (e.g., 6G communication system), converged system of multiple access systems, or evolved system.

[0068] The terminal devices involved in the embodiments of this application may include various access terminals, mobile devices, user terminals, or user equipment (UE) with wireless communication functions. For example, terminal devices may be mobile phones, tablets, computers with wireless transceiver capabilities, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, machine-type communication (MTC) terminals, customer premises equipment (CPE), wireless terminals in self-driving, remote medical, smart grid, transportation safety, smart city, and smart home applications. The embodiments of this application do not limit the application scenarios. In this application, the aforementioned terminal devices and chips that can be configured on the aforementioned terminal devices are collectively referred to as terminal devices.

[0069] By way of example and not limitation, in this embodiment, the terminal device can also be a wearable device. Wearable devices, also known as wearable smart devices, are a general term for devices that utilize wearable technology to intelligently design and develop everyday wearables, such as glasses, gloves, watches, clothing, and shoes. Wearable devices are portable devices that are worn directly on the body or integrated into the user's clothing or accessories. Wearable devices are not merely hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are feature-rich, large in size, and can achieve complete or partial functions without relying on a smartphone, such as smartwatches or smart glasses, as well as those that focus on a specific type of application function and require the use of other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0070] This application relates to federated learning. Federated learning (FL) is a distributed machine learning technique or framework. Currently, federated learning allows data to remain local, using encrypted parameter exchange to build a public model without violating data privacy regulations. This public model can achieve the same model performance (such as accuracy and recall) as a model trained by aggregating data. However, during the construction of the public model, the data itself does not move, nor does it leak privacy or affect data compliance. Thus, the built models serve only their local targets within their respective regions.

[0071] Federated learning can be divided into horizontal federated learning and vertical federated learning, as follows: Figure 1 and Figure 2 As shown. In federated learning, a horizontal row of a data matrix (e.g., a table) represents a training sample, and a vertical column represents a data feature (or label). Horizontal federated learning is essentially the jointing of samples. It is suitable for situations where there is significant overlap in the data features of participants but minimal overlap in sample labels. It can combine multiple rows of samples with the same features from multiple participants for federated learning; that is, the training data of each participant is horizontally partitioned, increasing the total number of training samples. Vertical federated learning is essentially the jointing of features. It is suitable for situations where there is significant overlap in the training sample labels of participants but minimal overlap in data features. It can combine different data features of common samples from multiple participants for federated learning; that is, the training data of each participant is vertically partitioned.

[0072] Figure 3 This diagram illustrates the training process of horizontal federated learning. As you can see, horizontal federation consists of a central server node and multiple edge client nodes. The original data is distributed across the client nodes. The server node does not have the original data, and the client nodes are not allowed to send the original data used for training to the server node.

[0073] First, assuming there are K clients in total, the dataset on each client node can be represented in the following form:

[0074]

[0075] Where x represents the sample data and y represents the corresponding label data. In horizontal federated learning, each sample data includes a label, meaning the label and data are stored together.

[0076] Next, the data analysis module on each client node can train its own model based on the linear regression algorithm, which is called a sub-model or local model.

[0077]

[0078] Assuming the loss function used in linear regression is the mean squared error (MSE), then the objective function for training each sub-model is as follows:

[0079]

[0080] Where, n I Let be the number of samples in the i-th client.

[0081] It should be understood that the entire training process involves making the loss function L... I Minimum.

[0082] Taking a linear regression model as an example, the training process can be divided into the following 5 steps:

[0083] Step 1: Each client node generates the gradient of the sub-model.

[0084]

[0085] Step 2: Each client reports the number of samples and its local gradient value to the server, i.e., n I as well as

[0086] Step 3: After the server receives the number of samples and the gradient values ​​of the sub-model from each client, the server aggregates the gradient values:

[0087]

[0088] Where, ||K| represents the number of clients, and P I =n I / ∑ I n I .

[0089] Step 4: The server distributes the aggregated gradients to each client node participating in the training, and then the client nodes update the model parameters locally.

[0090]

[0091] That is, each client node will local Θ I Perform the replacement. Here, α is the training optimization step size.

[0092] Step 5: After the client node updates the sub-model, the client node calculates the loss function L.I The value of . If training is not yet complete, the client node proceeds to step 1 to continue the horizontal federated training process.

[0093] After training, each client node retains a local copy of the model for local inference.

[0094] It should be understood that in the above training process, the server node can control the end of training by the number of iterations, such as terminating training after 10,000 iterations, or by setting a threshold for the loss function, for example, L. I Training ends when the result is ≤0.0001, or when the difference between two consecutive training results is less than 0.0001, etc. Other methods of indicating the end of training are not limited in this application.

[0095] It should be noted that, in this application, the client's model information refers to the number of samples and the local gradient value determined by the client based on local data, i.e., n I as well as The model information of the edge server refers to the gradient information obtained after aggregating the model information reported by the client, as well as the number of samples corresponding to the model information obtained within the coverage area of ​​the edge server. The target model information of the cloud server refers to the gradient information obtained after aggregating the model information directly reported by the client and the model information reported by the edge server.

[0096] Next, we will introduce three different horizontal federated learning architectures. Figure 4 For a horizontal federated learning architecture in cloud-client mode, Figure 5 For edge-client mode horizontal federated learning architecture, Figure 6 It is a horizontal federated learning architecture in the cloud-edge-client mode.

[0097] It should be noted that a horizontal federation includes a coordinator and several participants. The participants are responsible for training the model using local data, while the coordinator is responsible for aggregating the models from all the participants.

[0098] First, in such Figure 4In the cloud-client model shown, the total number of participating clients can reach millions, providing the massive datasets required for deep learning. However, the models trained locally on the clients can be very large. For example, taking a neural network model as an example, assuming 1000 input parameters, 200 hidden layers, and 500 nodes per hidden layer, the size of the neural network model trained by each client could reach 100MB. Due to network congestion and other reasons, communication between the client and the cloud server becomes slow and unpredictable. If there is network congestion, the model may take a long time to be transmitted to the cloud, resulting in extremely low training efficiency. In addition, the data distribution of each client is inconsistent, leading to inconsistent training durations and affecting the overall duration of horizontal federated training.

[0099] In such Figure 5 In the edge-client mode shown, this approach effectively mitigates the impact on cloud servers caused by client-reported models. Specifically, after training a model, the client can report it to a local edge server, which then aggregates the reported sub-models locally before sending the aggregated model to the client for inference. However, since each edge server can only connect to a limited number of clients, insufficient data inevitably leads to a loss of training performance.

[0100] like Figure 6 The cloud-edge-client model shown is a compromise between the two models above. The client first uploads the sub-models to the edge server. The edge server aggregates the sub-models locally and then uploads the aggregated model to the cloud server. The cloud server finally aggregates the intermediate model from all the edge servers.

[0101] The system architecture that can be applied to this application is mainly a system architecture with a hybrid horizontal federated training mode, such as... Figure 7 As shown. In Figure 7 In this process, the client and the cloud server can report sub-models through either the cloud-client mode or the cloud-edge-client mode.

[0102] Furthermore, this application can also be extended to other system architectures, such as... Figure 8 The diagram illustrates horizontal federated learning in a 5G network architecture based on NWDAF. In this model, the lower-layer network data analytics function (NWDAF) and the middle-layer NWDAF are participants in the horizontal federated process, while the upper-layer NWDAF is the coordinator.

[0103] Alternatively, this application can be extended to... Figure 9The example shown is a horizontal federation learning scenario in the NWDAF-RAN scenario. In this scenario, multiple UEs are horizontally federated, with the UE and RAN being the participants in the horizontal federation process, and the NWDAF acting as the coordinator in the horizontal federation process.

[0104] It should be understood that this application may also contain similar... Figures 7 to 9 Other system architectures or other scenarios under this application, namely, the participants and coordinators of a horizontal federation satisfying the above three inclusion relationships, should all be within the scope of protection of this application, and this application does not limit them.

[0105] Figure 10 A schematic flowchart of the federated learning training method provided in this application is shown. The method 1000 includes:

[0106] S1001, Obtain the client's first capability information and second capability information.

[0107] It should be noted that, in this embodiment, the first capability information and the second capability information of the client can be obtained by a cloud server, an edge server, or other third-party servers, and this application is not limited to any particular server.

[0108] Specifically, the first capability information may include one or more of the following: memory information, hard disk information, computing power information, load information, and channel bandwidth information. It should be understood that this first capability information is dynamic; in other words, it changes at different times. This first capability information may change due to differences in the client's location, network channel, environmental factors, or business scenarios. It is worth clarifying that the first capability information refers to the first capability information currently available to the client; for example, memory information refers to the memory information currently available to the client.

[0109] Specifically, the second capability information may include one or more of the following: client capabilities for federated learning, federated learning algorithm type, and region information. It should be understood that this second capability information is inherent to the client; that is, it will not change due to changes in the client's environment, network channel conditions, or business scenario. Furthermore, the second capability information is not limited to the capabilities listed above. It may be pre-stored in the server, for example, in the server's pre-configuration information. Alternatively, it may be received from an external device. For example, the server may send a request to the client to request the client's second capability information. When the client receives the request, it returns the second capability information to the server in a response message. This application does not limit the method for obtaining the second capability information.

[0110] S1002, the target server is determined based on the first capability information and the second capability information.

[0111] Specifically, based on the first capability information, the server determines the recipient of the model information reported by the client participating in federated training, i.e., whether the client reports the model information directly to the cloud server or to the edge server.

[0112] It should be understood that, as described above, this first capability information is dynamic. Therefore, the server determines the recipient of the model information to be reported based on the client's first capability information during the training period. For example, if the server determines that the client's first capability information includes memory greater than or equal to 1GB, hard disk greater than or equal to 10GB, the number of graphics processing units (GPUs) in the computing power greater than or equal to 4, the load greater than or equal to 3, and the bandwidth between the client and the cloud server greater than or equal to 5Mbps, the server determines that the client can directly report the model information to the cloud server. Conversely, if one or more parameters in the client's first capability information are less than the above values, the server determines that the client should report the model information to the edge server.

[0113] It should be noted that the above values ​​can be set differently during different training sessions. When the client's first capability information is greater than or equal to this value, it indicates that the client has a low requirement for training bandwidth or has good channel quality with the cloud server. It is suitable to directly report model information to the cloud server without having to report model information to the edge server first, and then report the aggregated model to the cloud server after aggregation by the edge server, thereby improving the efficiency of federated training.

[0114] In addition, the server will further select clients to participate in training based on the obtained second capability information. For example, in a certain federated training, the server uses the linear regression algorithm for federated training, then clients that do not support the linear regression algorithm will not participate in that federated training.

[0115] S1003, send a first instruction message to the client, which instructs the client to report model information to the cloud server or edge server.

[0116] Specifically, when the server determines, according to step 1002, that the client is reporting model information to the cloud server, the server sends a first indication message to the client, which may include the address information of the cloud server. When the server determines, according to step 1002, that the client is reporting model information to the edge server, the server sends a first indication message to the client, which may include the address information of the edge server.

[0117] Therefore, the model training method provided in this application allows the server to dynamically determine whether the client should report model information to the cloud server or the edge server based on the client's capability information, making the client's reporting of model information more flexible in a horizontal federated learning training system, thereby improving the efficiency of federated training.

[0118] It is worth noting that the address information of the client, cloud server, or edge server involved in this invention may be one or more of the following information corresponding to the client, cloud server, or edge server: identification information, Internet Protocol IP address, tunnel endpoint identifier (TEID), Uniform Resource Locator (URL), fully qualified domain name (FQDN), or other descriptions involving address information, which can be referred to here and will not be repeated here.

[0119] according to Figure 10 Method 1000 shown Figure 11 A schematic flowchart of the federated learning training method 1100 provided in this application is shown, which further includes:

[0120] The server obtains the cloud server's requirement information, which includes the number of clients and / or data length. At this point, the server determines the method for clients to report model information based on the obtained first capability information, second capability information, and requirement information.

[0121] It should be understood that the server can obtain the first and second capability information of multiple clients. In this case, since the training time is limited and the training results do not change significantly with the number of clients participating in the training, the server can pre-set the number of clients participating in the training and the length of data for each client before training, i.e., the requirement information, so as to avoid the situation of low training efficiency caused by ineffective training process.

[0122] It should be understood that, in this embodiment, with Figure 10 The same steps are repeated here for simplicity.

[0123] Next, the servers are cloud servers ( Figure 12 ) and edge servers ( Figure 13 This section will provide a detailed explanation of the implementation of this solution.

[0124] Figure 12A schematic flowchart of the federated learning training method 1200 provided in this application is shown. This method determines the way the client reports model information through a cloud server, which can effectively improve the efficiency of horizontal federated training.

[0125] S1201 (S1201a, S1201b), Client-specific capability registration.

[0126] Specifically, the client sends a request to the cloud server to register the client's inherent capabilities with the cloud server. These inherent capabilities (secondary capability information) may include, for example, the client's ability to support horizontal federated learning, the types of horizontal federated learning algorithms supported by the client, and the client's location information.

[0127] Optionally, the request information may be carried in an inherent capability registration request message or other messages, and this application does not impose any limitations on this.

[0128] It should be noted that this inherent capability can be understood as a static and unchanging capability, meaning that the inherent capability information of the client will not change dynamically over time.

[0129] S1202 (S1202a, S1202b), Client-inherent capability registration response.

[0130] Specifically, after receiving the client's inherent capability information, the cloud server sends a response message to the client, which notifies the client that its inherent capability information has been successfully registered.

[0131] Optionally, the response message can be a client-inherent capability to register a response message.

[0132] S1203 (S1203a, S1203b), registration of inherent capabilities of edge servers.

[0133] Specifically, the edge server sends a request to the cloud server to register its inherent capabilities with the cloud server. These inherent capabilities may include, for example, the types of horizontal federated learning algorithms it supports, the capabilities of the edge server supporting horizontal federation, its service area, and a list of client identifiers within that service area.

[0134] Optionally, the request information may be carried in an inherent capability registration request message or other messages, and this application does not impose any limitations on this.

[0135] S1204 (S1204a, S1204b) is the registration response for the inherent capabilities of the edge server.

[0136] Specifically, after receiving the inherent capability information of the edge server, the cloud server sends a response message to the edge server, which is used to notify the edge server that its inherent capability information has been successfully registered.

[0137] Optionally, the response message can be a registration response message inherent to the edge server's capabilities.

[0138] S1205 (S1205a, S1205b), Client-side variable capability query.

[0139] Specifically, the cloud server sends a query request to the client, which is used to query the client's variable capabilities (first capability information). These variable capabilities may include, for example, memory, hard drive, computing power, load, and channel bandwidth between the client and the cloud server.

[0140] Optionally, the query request information may be carried in a variable capability query request message or other messages, and this application does not impose any limitations on this.

[0141] It should be understood that the cloud server can query the client for dynamic capability information over multiple time periods.

[0142] S1206 (S1206a, S1206b), Client variable capability query response.

[0143] Specifically, after receiving the variable capability query information, the client sends a response message to the cloud server, which is used to report the client's variable capability information.

[0144] Optionally, the response message can be a client variable capability query response message.

[0145] It should be understood that since the variable capability information of the client queried by the cloud server can be the variable capability information of the client within multiple time periods, the client needs to report the variable capability information within these multiple time periods.

[0146] S1207 (S1207a, S1207b), edge server variable capability query.

[0147] Specifically, the cloud server sends a query request to the edge server to inquire about the edge server's variable capabilities. These variable capabilities may include, for example, memory, hard drive space, computing power, load, and channel bandwidth between the edge server and the cloud server.

[0148] Optionally, the query request information may be carried in a variable capability query request message or other messages, and this application does not impose any limitations on this.

[0149] Similarly, the cloud server can also query the variable capability information of the edge server for multiple time periods, similar to step 1204.

[0150] S1208 (S1208a, S1208b), edge server variable capability query response.

[0151] Specifically, after receiving the variable capability query information, the edge server sends a response message to the cloud server. This response message is used to report the variable capability information of the edge server.

[0152] Alternatively, the response message may be an edge server variable capability query response message.

[0153] Similarly, the cloud server queries the variable capability information of the edge server over multiple time periods. This time period is the period during which the cloud server prepares to conduct horizontal federated learning training, and it is the same time period as in step 1205 above. This preparation indicates that the cloud server can ultimately select one sub-time period from multiple time periods for horizontal federated learning training.

[0154] S1209, the cloud server determines that the first client reports model information to the cloud server, and the second client reports model information to the cloud server through the edge server.

[0155] Specifically, the cloud server determines the list of clients participating in the training based on the client variable capability information and edge server variable capability information obtained in steps 1206 and 1208, respectively, and determines whether each client in the list belongs to the first client or the second client.

[0156] It should be understood that when the cloud server determines the first client and the second client, it will compare the obtained variable capability information with the variable capability threshold of the client that it has stored. If any variable capability information is less than the threshold, it can be determined that it is the second client; otherwise, it is the first client.

[0157] For example, the cloud server can pre-store a client's memory threshold of 1GB, a hard disk threshold of 12GB, a computing power (such as the number of GPUs) threshold of 4, a load threshold of 3, and a bandwidth threshold of 5M / bps between the client and the cloud server.

[0158] It should be understood that the above thresholds are for illustrative purposes only and not as limitations. The thresholds pre-stored on the cloud server may vary depending on the characteristics of the device or other factors, and this application does not limit them.

[0159] It should be understood that the first client and the second client mentioned above represent a type of client. In an extreme case, when there is only one client of this type, it can also represent a single client. The first client represents the type of client that directly reports model information to the cloud server, while the second client represents the type of client that reports model information to the edge server, and the edge server aggregates the model information before reporting the aggregated model to the cloud server.

[0160] It should also be noted that when determining which clients need to participate in training, the cloud server can randomly select the required number of clients from the queried clients to participate in the final training. In other words, the cloud server will not necessarily use all queried clients for training, but will randomly select a subset of the queried clients.

[0161] S1210, the cloud server sends horizontal federated model training requests to the first client and the second client respectively.

[0162] Specifically, in S1210a, when the client is the first client, the horizontal federated model training request message sent by the cloud server may include at least one of the following information: algorithm information (such as linear regression), initial model file, training data length, and cloud aggregation point indication information.

[0163] The initial model file refers to Figure 3 The Θ values ​​used to generate the local model in each client during the lateral federated learning training process are shown below. A Θ B , ...Θ K The length of the training data refers to the number of data points used for training on each client, that is... Figure 3 n I The cloud aggregation point indication information is used to instruct the client to report the local gradient values ​​and training data length to the cloud server. This cloud aggregation point indication information is an example of the first indication information, and it may include the address information of the cloud server.

[0164] S1210b, when the client is a second client, the horizontal federated model training request message sent by the cloud server may include at least one of the following information: algorithm information (such as linear regression), initial model file, training data length, edge aggregation point indication information, and edge server identifier.

[0165] The initial model file and training data length are the same as described above. The edge aggregation point indication information is used to indicate that the client needs to report the model information to the edge server. It can be an example of the second indication information. The edge server identifier is used to locate or indicate the edge server to which the client reports the model information.

[0166] It should be understood that the identification information of the edge server is used to instruct the second client to report the model information to the edge server corresponding to the identification information.

[0167] S1211, Horizontal Federated Model Training Configuration Notification.

[0168] Specifically, regarding step 1210b, the cloud server can send a horizontal federated model training configuration notification to the edge server. This message can be an example of the third instruction information, and it may include the identification information of the second client, such as the address information of the second client. This notification message is used to inform the edge server to report the model information of the edge server. It should be understood that the edge server needs to wait for the second client to report the model information, aggregate the model information of the second client, and then report the aggregated model to the cloud server.

[0169] It should be noted that since cloud servers can communicate with multiple edge servers, the message can also carry the address information of the cloud server, which can be the Internet Protocol IP address and / or channel port number used by the cloud server to communicate with the edge server.

[0170] It should be understood that step 1211 can occur before or after step 1210; there is no necessary order between steps 1210 and 1211. However, it is important to note that these two steps must occur before step 1210. In other words, the training request and training configuration need to be completed before the client reports model information to the cloud server or edge service.

[0171] S1212 (S1212a, S1212b), Lateral Federated Training Response.

[0172] Specifically, in S1212a, the first client updates the gradient information locally. Training data length n I The client's model information is sent to the corresponding cloud server.

[0173] S1212b, the second client will update the gradient information locally. Training data length n I Send it to the corresponding edge server.

[0174] S1213, edge aggregation of local gradient information.

[0175] Specifically, the edge server aggregates the model information reported by the clients it is responsible for in this domain (i.e., the second client) based on the information reported in step 1212b. The aggregation formula remains the same as described above. Figure 3 Step 3 in the process will not be repeated here.

[0176] S1214, lateral federated model training response.

[0177] Specifically, the edge server aggregates the model information from the second client to generate aggregated gradient information, which is the edge server's model information. The edge server then reports this model information to the cloud server via a horizontal federated model training response message.

[0178] S1215, Aggregation of local gradient information.

[0179] Specifically, the cloud server performs a final aggregation of the model information from the first client and the model information reported by the edge server. The aggregation method can still be reused. Figure 3 Step 3, which is to obtain the aggregated gradient information That is, the target model information of the cloud server.

[0180] It is important to note that during this aggregation process, a client can only report model information in one way.

[0181] S1216 (S1216a, S1216b) is a cloud server that sends aggregated gradient information to the first client and the edge server.

[0182] Specifically, the cloud server sends the aggregated gradient information obtained in step 1215 to the first client and the edge server.

[0183] S1217, the edge server sends the aggregated gradient information to the second client.

[0184] Specifically, the edge server forwards the gradient information received from the cloud server to the second client.

[0185] S1218 (S1218a, S1218b) involves the first client and the second client updating model information.

[0186] Specifically, after the first client and the second client receive the aggregated gradient information sent by the cloud server, they then, according to the above... Figure 3 Step 4 in the process will local Θ I Replace it.

[0187] It should be understood that steps 1218a and 1218b may not be performed simultaneously. In other words, the first client can start the update process after receiving the aggregated gradient information sent by the cloud server, without having to perform the update operation at the same time as the second client.

[0188] It should be noted that, in this embodiment, the edge server refers to the edge server that serves the second client, meaning the second client is located within the service area of ​​the edge server. Since the second client can be located in different areas (there can be one or more), there can also be one or more edge servers.

[0189] In this embodiment, by using the variable capability information of the cloud server based on the client and the edge server, the method of the client reporting model information can be flexibly selected, which can alleviate the access pressure on the cloud server and improve the efficiency of horizontal federated training.

[0190] Figure 13 A schematic flowchart of the federated learning training method 1300 provided in this application is shown. This method determines the way the client reports model information through an edge server, which can effectively improve the efficiency of horizontal federated training.

[0191] S1301 (S1301a, S1301b), Client-specific capability registration request.

[0192] Specifically, each client in a horizontal federated learning environment within the edge server's coverage area will register its inherent capability information on the edge server. This information may include, for example, the client's capabilities that support horizontal federated learning, the types of horizontal federated learning algorithms supported by the client, and the client's location.

[0193] Optionally, the client can report its inherent capabilities by sending a first message to the edge server. This first message can be a client-inherent capability registration request message carrying the client's inherent capabilities.

[0194] S1302 (S1302a, S1302b), Client-inherent capability registration response.

[0195] Specifically, after the edge server receives the inherent capability information reported by the client within its coverage area, the edge server sends a response message to the client to indicate that the client's inherent capability registration was successful. Optionally, this response message can be a client inherent capability registration response message.

[0196] S1303 (S1303a, S1303b), edge server inherent capability registration request.

[0197] Specifically, each horizontal federated learning edge server registers its inherent capability information, such as the types of horizontal federated learning algorithms it supports, the edge server capabilities it supports, its service area, and a list of client identifiers within the service area, on the horizontal federated learning cloud server.

[0198] S1304 (S1304a, S1304b), edge server inherent capability registration response.

[0199] Specifically, the cloud server sends response messages to each edge server, which indicate that the cloud server has received the inherent capability information of the edge server.

[0200] Optionally, the response message can be a registration response message inherent to the edge server's capabilities.

[0201] S1305 (S1305a, S1305b), edge server variable capability query.

[0202] Specifically, the cloud server sends query requests to each edge server to inquire about the variable capabilities of the edge server. These variable capabilities may include, for example, memory, hard drive space, computing power, load, and channel bandwidth between the edge server and the cloud server.

[0203] Optionally, the query request information may be carried in an edge server variable capability query request message or other messages, and this application does not impose any limitations on this.

[0204] It should be understood that the variable capability information of the edge server queried by the cloud server can be the variable capability information of the edge server within a certain period of time. Ultimately, the cloud server can select a period of time within that period as the time for horizontal federated learning training.

[0205] S1306 (S1306a, S1306b), edge server variable capability query response.

[0206] Specifically, after receiving the variable capability query information sent by the cloud server, the edge server sends a response message to the cloud server. This response message is used to report the variable capability information of the edge server.

[0207] Alternatively, the response message may be an edge server variable capability query response message.

[0208] It should be understood that the variable capability information reported by the edge server should be the variable capability information within the time period queried by the cloud server in step 1305.

[0209] S1307, determine the number of clients aggregated for each edge server and / or the total data length.

[0210] Specifically, after obtaining the variable capability information of the edge server during the training period in step 1306, the cloud server determines the number of clients to be aggregated and / or the total data length for each edge server.

[0211] It should be noted that the number of clients requiring aggregation refers to the number of clients participating in training. Some of these clients may directly report model information to the cloud server, while others may report model information to the edge server. The edge server first aggregates the information before reporting the aggregated edge server model information to the cloud server, which then performs further aggregation. The total data length represents the sum of the data from the clients that the edge server needs to aggregate.

[0212] Furthermore, the premise that the cloud server can determine the number of clients aggregated and / or the total data length of each edge server is that the cloud server can determine the load, memory, computing power, or hard disk information required by each edge server based on the variable capacity information reported by the edge server.

[0213] S1308 (S1308a, S1308b), cloud server training request.

[0214] Specifically, the cloud server sends a horizontal federated training request to each edge server, including algorithm information, initial model, number of clients required, and / or length of data required.

[0215] S1309 (S1309a, S1309b), Client variable capability query request.

[0216] Specifically, the edge server sends a query request message to the client to query the client's variable capabilities, which may include memory, hard disk, computing power, load, and channel bandwidth between the client and the cloud server.

[0217] Optionally, the query request message can be a client-defined variable capability query request message.

[0218] It should be understood that the variable capacity in this step refers to the variable capacity within the time period queried by the cloud server.

[0219] S1310 (S1310a, S1310b), Client variable capability query response.

[0220] Specifically, the client reports its variable capabilities during the training period on the cloud server to its corresponding edge server via a response message.

[0221] Optionally, the response message can be a client variable capability query response message.

[0222] S1311 (S1311a, S1311b) determines the method by which the client reports model information.

[0223] Specifically, based on the requirement information from step 1308 and the information obtained in step 1310, each edge server determines which horizontal federated learning clients within its coverage area can directly report model information to the cloud server, for example... Figure 13 The first client under edge server A shown, or the edge server determining which lateral federated learning clients within its coverage area can aggregate through the local edge server to send the aggregated model to the cloud server, for example... Figure 13 The second client is shown under edge server B.

[0224] It should be noted that the process of determining this is different from... Figure 12 The process by which the cloud server determines the first and second clients is largely the same. The difference lies in that the threshold for variable capability information is pre-stored in the edge server in this embodiment. When the edge server determines that the variable capability information of a queried client is greater than the threshold, it identifies that client as an example of the first client; otherwise, it identifies it as an example of the second client. Furthermore, the number of clients participating in the training can be randomly selected from the queried clients based on the cloud server's needs.

[0225] It should be understood that edge server A can still have a client type that aggregates model information through edge server A before reporting the aggregated model to the cloud server; this is the second client. Similarly, edge server B can also have a first client. In other words, the edge server can determine the number of first and second clients under it based on the needs of the cloud server and the variable capabilities of the clients. Therefore, Figure 13 Other steps are not shown for the sake of clarity or ease of description. Those skilled in the art will clearly understand that this figure is illustrative and not limiting.

[0226] S1312, Lateral Federation Training Configuration Notification.

[0227] Specifically, each edge server sends a horizontal federated training configuration notification to the cloud server, which informs the cloud server that it needs to wait for the first client to report model information. This notification message may include the identification information of the first client.

[0228] It should be understood that when a client exists within the scope of the edge server, the edge server needs to send this configuration notification message to the cloud server so that the cloud server can clearly know which clients are directly reporting model information, thereby saving training time and improving training efficiency.

[0229] It should be understood that Figure 13 Step 1312 is omitted for simplicity; the same operation in other clients is not shown. Figure 13This application should not be construed as limiting the implementation of this application.

[0230] S1313 (S1313a, S1313b), edge server training request.

[0231] Specifically, the edge server sends a horizontal federated training request to the client. For the first client, in step 1313a, the request message sent by the edge service may include: algorithm information (such as linear regression), initial model file, training data length, cloud aggregation point indication information, and cloud server identifier. For the second client, in step 1313b, the request message sent by the edge service may include: algorithm information (such as linear regression), initial model file, training data length, and edge aggregation point indication information.

[0232] S1314 (S1314a, S1314b), Lateral Federated Training Response.

[0233] Specifically, in step 1314a, the first client will transfer the local gradient information. Training data length n I Send it to the corresponding cloud server. Step 1314b: The second client sends the local gradient information. Training data length n I Send it to the corresponding edge server.

[0234] S1315, edge aggregation of local gradient information.

[0235] Specifically, after receiving the client-side model information (i.e., local gradient information and training length) from the second client in step 1314, the edge server first utilizes the local gradient information... Figure 3 Step 3 involves aggregation.

[0236] S1316, Training Response.

[0237] Specifically, the edge server reports its aggregated model information to the cloud server. This model information can be carried in the training response message.

[0238] S1317, Aggregation of local gradient information.

[0239] Specifically, the cloud server performs a final aggregation of the model information from the first client and the model information reported by the edge server. The aggregation method can still be reused. Figure 3 Step 3, which is to obtain the aggregated gradient information

[0240] S1318 (S1318a, S1318b) is where the cloud server sends aggregated gradient information to the first client and the edge server.

[0241] Specifically, the cloud server sends the aggregated gradient information obtained in step 1015 to the first client and the edge server.

[0242] S1319, the edge server sends the aggregated gradient information to the second client.

[0243] Specifically, the edge server forwards the gradient information received from the cloud server to the second client.

[0244] S1320 (S1320a, S1320b) involves the first client and the second client updating model information.

[0245] Specifically, after the first client and the second client receive the aggregated gradient information sent by the cloud server, they then, according to the above... Figure 3 Step 4 in the process will local Θ I Replace it.

[0246] It should be understood that steps 1320a and 1320b may not be performed simultaneously. In other words, the first client can start the update process after receiving the aggregated gradient information sent by the cloud server, without having to perform the update operation at the same time as the second client.

[0247] It should be noted that, in this embodiment, the edge server refers to the edge server that serves the second client, meaning the second client is located within the service area of ​​the edge server. Since the second client can be located in different areas (there can be one or more), there can also be one or more edge servers.

[0248] In this embodiment, by using the edge server to flexibly select the method for the client to report model information based on the variable capability information of the client and the needs of the cloud server, the access pressure on the cloud server can be alleviated and the efficiency of horizontal federated training can be improved.

[0249] The above, combined with Figures 10 to 13 The methods provided in the embodiments of this application are described in detail below. Figures 14 to 17 This application provides a detailed description of the model training apparatus provided in its embodiments.

[0250] Figure 14 This is a schematic block diagram of a model training apparatus provided in an embodiment of this application. As shown in the figure, the model training apparatus 10 may include a processing module 11 and a transceiver module 12.

[0251] In one possible design, the model training device 10 may correspond to the server in the above method embodiment.

[0252] Specifically, the model training apparatus 10 may correspond to the server in methods 1000, 1100, 1200, and 1300 according to embodiments of this application, and the model training apparatus 10 may include tools for performing... Figure 10 Method 1000 or Figure 11 Method 1100 or Figure 12 Method 1200 or Figure 13 The module executing the server method in method 1300. Furthermore, each unit in the model training device 10 and the aforementioned other operations and / or functions are respectively for implementing... Figure 10 Method 1000 or Figure 11 Method 1100 or Figure 12 Method 1200 or Figure 13 The corresponding process of method 1300 in the middle.

[0253] Among them, when the model training device 10 is used to perform Figure 10 When performing method 1000, processing module 11 can be used to execute steps 1001 and 1002 in method 1000. Transceiver module 12 can be used to execute step 1003 in method 1000.

[0254] When the model training device 10 is used to perform Figure 11 When using method 1100, processing module 11 can be used to execute steps 1101 and 1102 in method 1100. Transceiver module 12 can be used to execute step S1103 in method 1100.

[0255] When the model training device 10 is used to perform Figure 12 When performing method 1200, processing module 11 can be used to execute steps 1209 and 1215 in method 1200. Transceiver module 12 can be used to execute steps 1201 to 1208, as well as steps 1210, 1211, 1214, and 1216 in method 1200.

[0256] When the model training device 10 is used to perform Figure 13 When performing method 1300, processing module 11 can be used to execute steps 1311 and 1315 in method 1300. Transceiver module 12 can be used to execute steps 1301 to 1330, 1306, 1308 to 1310, 1312 to 1314, 1316, 1318, and 319 in method 1300.

[0257] Figure 15This is a schematic block diagram of the model training apparatus provided in an embodiment of this application. As shown in the figure, the model training apparatus 20 may include a transceiver module 21 and a processing module 22.

[0258] In one possible design, the model training device 20 may correspond to the cloud server in the above method embodiment, or a chip configured in the cloud server.

[0259] Specifically, the model training device 20 may correspond to the server in method 1000, method 1100, and method 1200 according to embodiments of this application, and the model training device 20 may include tools for performing... Figure 10 Method 1000 or Figure 11 Method 1100 or Figure 12 The module executing the server method in method 1200. Furthermore, each unit in the model training device 20 and the aforementioned other operations and / or functions are respectively for implementing... Figure 10 Method 1000 or Figure 11 Method 1100 or Figure 12 The corresponding process of method 1200 in the middle.

[0260] When the model training device 20 is used to perform Figure 10 When using method 1000, the transceiver module 21 can be used to execute step 1003 in method 1000. The processing module 22 can be used to execute steps 1001 and 1002 in method 1000.

[0261] When the model training device 20 is used to perform Figure 11 When using method 1100, the transceiver module 21 can be used to execute step 1103 in method 1100. The processing module 22 can be used to execute steps 1101 and 1102 in method 1100.

[0262] When the model training device 20 is used to perform Figure 12 When performing method 1200, processing module 11 can be used to execute steps 1209 and 1215 in method 1200. Transceiver module 12 can be used to execute steps 1201 to 1208, as well as steps 1210, 1211, 1214, and 1216 in method 1200.

[0263] Figure 16 This is a schematic block diagram of the model training apparatus provided in an embodiment of this application. As shown in the figure, the model training apparatus 30 may include a processing module 31 and a transceiver module 32.

[0264] In one possible design, the model training device 30 may correspond to the edge server in the above method embodiment.

[0265] Specifically, the model training device 30 may correspond to the server in method 1000, method 1100, and method 1300 according to embodiments of this application, and the model training device 30 may include tools for performing... Figure 10 Method 1000 or Figure 11 Method 1100 or Figure 13 The module executing the server method in method 1300. Furthermore, each unit in the model training device 30 and the aforementioned other operations and / or functions are respectively for implementing... Figure 10 Method 1000 or Figure 11 Method 1100 or Figure 13 The corresponding process of method 1300 in the middle.

[0266] Among them, when the model training device 30 is used to perform Figure 10 When performing method 1000, processing module 31 can be used to execute steps 1001 and 1002 in method 1000. Transceiver module 32 can be used to execute step 1003 in method 1000.

[0267] When the model training device 30 is used to perform Figure 11 When performing method 1100, processing module 31 can be used to execute steps 1101 and 1102 in method 1100. Transceiver module 32 can be used to execute step S1103 in method 1100.

[0268] When the model training device 30 is used to perform Figure 13 When performing method 1300, processing module 31 can be used to execute steps 1311 and 1315 in method 1300. Transceiver module 32 can be used to execute steps 1301 to 1330, 1306, 1308 to 1310, 1312 to 1314, 1316, 1318, and 319 in method 1300.

[0269] According to the aforementioned method, Figure 17 A schematic diagram of the model training device 40 provided in the embodiments of this application is shown below. Figure 17 As shown, the device 40 can be a model training device.

[0270] The device 40 may include a processor 41 (i.e., an example of a processing module) and a memory 42. The memory 42 is used to store instructions, and the processor 41 is used to execute the instructions stored in the memory 42 to cause the device 40 to perform, for example... Figure 10 , Figure 11 or Figure 12 or Figure 13 The steps executed by the server in the corresponding method.

[0271] Furthermore, the device 40 may also include an input port 43 (i.e., an example of a transceiver module) and an output port 44 (i.e., another example of a transceiver module). Furthermore, the processor 41, memory 42, input port 43, and output port 44 can communicate with each other through internal connection paths to transmit control and / or data signals. The memory 42 is used to store computer programs, and the processor 41 can be used to call and run the computer program from the memory 42 to control the input port 43 to receive signals and control the output port 44 to send signals, thus completing the server steps in the above method. The memory 42 can be integrated into the processor 41 or disposed separately from the processor 41.

[0272] Optionally, the input port 43 can be a receiver, and the output port 44 can be a transmitter. The receiver and transmitter can be the same or different physical entities. When they are the same physical entity, they can be collectively referred to as transceivers.

[0273] Optionally, if the model training device 40 is a chip or circuit, the input port 43 is an input interface and the output port 44 is an output interface.

[0274] As one implementation method, the functions of input port 43 and output port 44 can be implemented using transceiver circuits or dedicated transceiver chips. Processor 41 can be implemented using dedicated processing chips, processing circuits, processors, or general-purpose chips.

[0275] As another implementation method, the model training device provided in this application embodiment can be implemented using a general-purpose computer. The program code that implements the functions of processor 41, input port 43, and output port 44 is stored in memory 42, and the general-purpose processor implements the functions of processor 41, input port 43, and output port 44 by executing the code in memory 42.

[0276] Each unit or unit in the model training device 40 can be used to execute the actions or processes performed by the server in the above method. Here, to avoid redundancy, its detailed description is omitted.

[0277] For the concepts, explanations, detailed descriptions, and other steps related to the technical solutions provided in the embodiments of this application involved in the device 40, please refer to the descriptions of these contents in the foregoing methods or other embodiments, which will not be repeated here.

[0278] According to the aforementioned method, Figure 18 A schematic diagram of the model training device 50 provided in the embodiments of this application is shown below. Figure 18 As shown, the device 50 can be a cloud server.

[0279] The device 50 may include a processor 51 (i.e., an example of a processing module) and a memory 52. ​​The memory 52 is used to store instructions, and the processor 51 is used to execute the instructions stored in the memory 52 to cause the device 50 to perform, as... Figure 10 , Figure 11 or Figure 12 or Figure 13 The steps performed by the terminal device.

[0280] Furthermore, the device 50 may also include an input port 53 (i.e., an example of a transceiver module) and an output port 54 (i.e., another example of a transceiver module). Furthermore, the processor 51, memory 52, input port 53, and output port 54 can communicate with each other through internal connection paths to transmit control and / or data signals. The memory 52 is used to store computer programs, and the processor 51 can be used to call and run the computer program from the memory 52 to control the input port 53 to receive signals and control the output port 54 to send signals, thus completing the steps of the terminal device in the above method. The memory 52 may be integrated into the processor 51 or may be disposed separately from the processor 51.

[0281] Optionally, the input port 53 can be a receiver, and the output port 54 can be a transmitter. The receiver and transmitter can be the same or different physical entities. When they are the same physical entity, they can be collectively referred to as transceivers.

[0282] Optionally, if the model training device 50 is a chip or circuit, the input port 53 is an input interface and the output port 54 is an output interface.

[0283] As one implementation method, the functions of input port 53 and output port 54 can be implemented using transceiver circuits or dedicated transceiver chips. Processor 51 can be implemented using dedicated processing chips, processing circuits, processors, or general-purpose chips.

[0284] As another implementation method, the cloud server provided in this application embodiment can be implemented using a general-purpose computer. The program code that implements the functions of processor 51, input port 53, and output port 54 is stored in memory 52. ​​The general-purpose processor implements the functions of processor 51, input port 53, and output port 54 by executing the code in memory 52.

[0285] The modules or units in the model training device 50 can be used to execute the actions or processes performed by the cloud server in the above method. Here, to avoid redundancy, their detailed descriptions are omitted.

[0286] For the concepts, explanations, detailed descriptions, and other steps related to the technical solutions provided in the embodiments of this application involved in the device 50, please refer to the descriptions of these contents in the foregoing methods or other embodiments, which will not be repeated here.

[0287] The terms “component,” “module,” “system,” etc., used in this specification are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0288] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0289] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0290] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0291] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0292] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0293] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0294] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training a model, characterized in that, include: Obtain the client's primary and secondary capability information; Based on the first capability information and the second capability information, the target server is determined to be a cloud server or an edge server, and the target server is the server on which the client reports model information; Send an instruction message to the client, the instruction message being used to instruct the client to report the client's model information to the target server.

2. The method according to claim 1, characterized in that, The first capability information includes at least one of the following information of the client: Memory information, hard disk information, computing power information, load information, channel bandwidth information, or, The second capability information includes at least one of the following information of the client: Federated learning client capabilities, federated learning algorithm types, and region information.

3. The method according to claim 1, characterized in that, The method further includes: Obtain the cloud server's requirement information, which includes the number of clients and / or data length; The step of determining the target server based on the first capability information and the second capability information includes: The target server is determined based on the first capability information, the second capability information, and the requirement information.

4. The method according to any one of claims 1 to 3, characterized in that, The indication information also includes the address information of the cloud server and / or the address information of the edge server.

5. A method for training a model, characterized in that, include: The cloud server receives model information from the first client and model information from the edge server, wherein the model information from the edge server is determined based on the model information from the second client; The cloud server determines the target model information based on the model information of the first client and the model information of the edge server; The cloud server sends the target model information to the first client and the edge server.

6. The method according to claim 5, characterized in that, The method further includes: The cloud server sends a first instruction message to the first client, which instructs the first client to report its model information to the cloud server.

7. The method according to claim 6, characterized in that, The first indication information includes the address information of the cloud server.

8. The method according to claim 5, characterized in that, The method further includes: The cloud server sends a second instruction to the second client, which instructs the second client to report its model information to the edge server.

9. The method according to claim 8, characterized in that, The second indication information includes the address information of the edge server.

10. The method according to any one of claims 5 to 9, characterized in that, The method further includes: The cloud server sends a third instruction to the edge server, the third instruction being used to instruct the edge server to report its model information to the cloud server.

11. The method according to claim 10, characterized in that, The third indication information includes the address information of the second client and / or the address information of the cloud server.

12. The method according to claim 5, characterized in that, The method further includes: The cloud server sends demand information to the edge server. The demand information includes the sum of the number of the first client and the second client and / or the sum of the data length. The demand information is used to determine the first client and the second client.

13. A model training device, characterized in that, include: The processing module is used to obtain the first capability information and the second capability information of the client, and determine the target server as a cloud server or an edge server based on the first capability information and the second capability information, wherein the target server is the server on which the client reports model information; The transceiver module is used to send instruction information to the client, which instructs the client to report its model information to the target server.

14. The apparatus according to claim 13, characterized in that, The first capability information includes at least one of the following information of the client: Memory information, hard disk information, computing power information, load information, channel bandwidth information, or, The second capability information includes at least one of the following information of the client: Federated learning client capabilities, federated learning algorithm types, and region information.

15. The apparatus according to claim 13, characterized in that, The processing module is also used for: Obtain the cloud server's requirement information, which includes the number of clients and / or data length, and specifically use it to determine the target server based on the first capability information, the second capability information, and the requirement information.

16. The apparatus according to any one of claims 13 to 15, characterized in that, The indication information also includes the address information of the cloud server and / or the address information of the edge server.

17. A device for model training, characterized in that, include: The transceiver module is used to receive model information from the first client and model information from the edge server, wherein the model information from the edge server is determined based on the model information from the second client. The processing module is used to determine the target model information based on the model information of the first client and the model information of the edge server; The transceiver module is also used to send the target model information to the first client and the edge server.

18. The apparatus according to claim 17, characterized in that, The transceiver module is also used for: Send a first instruction message to the first client, the first instruction message being used to instruct the first client to report the model information of the first client to the device.

19. The apparatus according to claim 18, characterized in that, The first indication information includes the address information of the device.

20. The apparatus according to claim 17, characterized in that, The transceiver module is also used for: Send a second instruction message to the second client, the second instruction message being used to instruct the second client to report the model information of the second client to the edge server.

21. The apparatus according to claim 20, characterized in that, The second indication information includes the address information of the edge server.

22. The apparatus according to any one of claims 17 to 21, characterized in that, The transceiver module is also used for: A third instruction message is sent to the edge server, the third instruction message being used to instruct the edge server to report the model information of the edge server to the device.

23. The apparatus according to claim 22, characterized in that, The third indication information includes the address information of the second client and / or the address information of the device.

24. The apparatus according to claim 17, characterized in that, The transceiver module is also used for: Send demand information to the edge server. The demand information includes the sum of the number of the first client and the second client and / or the sum of the data length. The demand information is used to determine the first client and the second client.

25. A model training device, characterized in that, include: A processor and a storage medium, the storage medium storing instructions that are executed by the processor. Cause the processor to perform the method as described in any one of claims 1 to 4, or This causes the processor to perform the method as described in any one of claims 5 to 12.

26. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when run on a computer,... Cause the computer to perform the method as described in any one of claims 1 to 4, or This causes the computer to perform the method as described in any one of claims 5 to 12.

27. A chip system, characterized in that, include: A processor is used to retrieve and run computer programs from memory. This causes the model training device equipped with the chip system to perform the method as described in any one of claims 1 to 4, or This causes the model training device equipped with the chip system to perform the method as described in any one of claims 5 to 12.

Images