Federated training methods, data analysis methods and devices, media and electronic equipment

Abstract

This disclosure relates to the field of computer technology, specifically to a federated training method, a data analysis method, a federated training device, a data analysis device, a medium, and an apparatus. The method includes: dividing multiple federated training terminals into multiple federated training groups; obtaining a model to be trained based on terminal feature information and the multiple federated training groups; distributing the backbone network and the corresponding head network of each federated training terminal to each federated training terminal; receiving model training parameters returned by the multiple federated training terminals, aggregating the multiple model training parameters to obtain an updated model to be trained; distributing the updated model to be trained to each federated training terminal; and repeating the above steps until the training process of the model to be trained meets the convergence condition. The technical solution of this disclosure can solve the problem of poor overall model integrity obtained by federated training in the prior art.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a federated training method, a data analysis method, a federated training device, a data analysis device, a computer-readable storage medium, and an electronic device. Background Technology

[0002] With the rapid development of communication services, it is often necessary to analyze user data in order to provide better services.

[0003] In related technologies, when a data analysis task needs to be performed, user data corresponding to the data analysis task can be collected from the terminal device and uploaded to the data analysis network element. The process of model training in the data analysis network element can be achieved through multiple terminal devices without leaving the local user data.

[0004] However, in related technologies, the terminals participating in model training often vary significantly. For example, from a geographical distribution perspective, some terminals are located in cities, while others are located in rural areas. Significant differences between terminals can lead to slower convergence of federated learning or poor overall model consistency, resulting in poor performance in data analysis.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this disclosure is to provide a federated training method, a federated training device, a computer-readable storage medium, and an electronic device, which can solve the problem of poor overall performance of models obtained through federated training in the prior art.

[0007] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0008] According to a first aspect of this disclosure, a federated training method is provided, comprising: acquiring terminal feature information of multiple federated training terminals; dividing the multiple federated training terminals into multiple federated training groups based on the terminal feature information; obtaining a model to be trained based on the terminal feature information and the multiple federated training groups; wherein the model to be trained includes a backbone network and head networks corresponding to the multiple federated training groups; distributing the backbone network and the head networks corresponding to each federated training terminal to each federated training terminal, so that each federated training terminal trains the backbone network and the head networks corresponding to each federated training terminal based on the terminal data of the federated training terminal, thereby obtaining model training parameters for each federated training terminal; receiving model training parameters returned by multiple federated training terminals, aggregating the multiple model training parameters to obtain an updated model to be trained; distributing the updated model to be trained to each federated training terminal; repeating the above steps until the training process of the model to be trained meets the convergence condition, and determining the model to be trained obtained by the last aggregation as the data analysis model for performing data analysis tasks.

[0009] In one exemplary embodiment of this disclosure, based on the aforementioned scheme, the model training parameters include the model training parameters of the backbone network and the model training parameters of the head network corresponding to each federated training group. The updated model to be trained is obtained by aggregating multiple model training parameters, including: aggregating the model training parameters of the backbone network and the model training parameters of the head network corresponding to each federated training group to obtain the updated model to be trained.

[0010] In an exemplary embodiment of this disclosure, based on the aforementioned scheme, the global training rounds of federated training are obtained; when the global training rounds of federated training meet preset conditions, the updated head networks corresponding to each federated training terminal are sent to each federated training terminal, so that each federated training terminal trains the head networks corresponding to each federated training terminal according to the terminal data of the federated training terminal, and obtains the model training parameters of each federated training terminal.

[0011] In one exemplary embodiment of this disclosure, based on the aforementioned scheme, multiple model training parameters are aggregated to obtain an updated model to be trained, including: aggregating the model training parameters of the head network corresponding to each federated training group to obtain an updated head network; and determining the updated model to be trained based on the fixed backbone network and the updated head network.

[0012] In one exemplary embodiment of this disclosure, based on the aforementioned scheme, when the global training rounds of federated training meet preset conditions, the updated head network corresponding to each federated training terminal is sent to each federated training terminal, including: determining the fixed rounds of the backbone network corresponding to the model to be trained; when the global training rounds of federated training are greater than the fixed rounds of the backbone network, the updated head network corresponding to each federated training terminal is sent to each federated training terminal.

[0013] According to a second aspect of this disclosure, a data analysis method is provided, comprising: receiving a data analysis request sent by a terminal device and determining the federated training group to which the terminal device belongs; wherein the data analysis request includes target data to be analyzed; inputting the target data into a data analysis model, and analyzing the target data through the backbone network in the data analysis model and the head network corresponding to the federated training group to obtain data analysis results; wherein the data analysis model is trained by the federated training method as described above.

[0014] According to a third aspect of this disclosure, a federated training apparatus is provided, comprising: a federated training group partitioning module, configured to acquire terminal feature information of multiple federated training terminals and partition the multiple federated training terminals into multiple federated training groups based on the terminal feature information; a model to be trained module, configured to obtain a model to be trained based on the terminal feature information and the multiple federated training groups; wherein the model to be trained includes a backbone network and head networks corresponding to the multiple federated training groups; a network distribution module, configured to distribute the backbone network and the head networks corresponding to each federated training terminal to each federated training terminal, so that each federated training terminal trains the backbone network and the head networks corresponding to each federated training terminal based on the terminal data of the federated training terminal, thereby obtaining model training parameters for each federated training terminal; a model aggregation module, configured to receive model training parameters returned by multiple federated training terminals and aggregate the multiple model training parameters to obtain an updated model to be trained; a distribution module, configured to distribute the updated model to be trained to each federated training terminal; and a model training module, configured to repeat the above steps until the training process of the model to be trained meets the convergence condition, and determine the model to be trained obtained by the last aggregation as the data analysis model for performing data analysis tasks.

[0015] According to a fourth aspect of this disclosure, a data analysis model analysis apparatus is provided, comprising: an analysis request receiving module, configured to receive a data analysis request sent by a terminal device and determine the federated training group to which the terminal device belongs; wherein the data analysis request includes target data to be analyzed; and a data analysis module, configured to input the target data into a data analysis model and analyze the target data through the backbone network in the data analysis model and the head network corresponding to the federated training group to obtain data analysis results; wherein the data analysis model is trained by a federated training method as described above.

[0016] According to a fifth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the federated training method of the first aspect or the data analysis method of the second aspect as described in the above embodiments.

[0017] According to a sixth aspect of this disclosure, an electronic device is provided, comprising:

[0018] One or more processors; and

[0019] A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the federated training method of the first aspect or the data analysis method of the second aspect as described in the above embodiments.

[0020] The technical solutions provided in this disclosure may have the following beneficial effects:

[0021] In one embodiment of the federated training method or data analysis method provided by this disclosure, terminal feature information of multiple federated training terminals can be obtained. Based on the terminal feature information, the multiple federated training terminals are divided into multiple federated training groups. A model to be trained is obtained based on the terminal feature information and the multiple federated training groups. The backbone network and the corresponding head network of each federated training terminal are distributed to each federated training terminal so that each federated training terminal trains the backbone network and the corresponding head network based on the terminal data of the federated training terminal, obtaining the model training parameters of each federated training terminal. The model training parameters returned by multiple federated training terminals are received, and the multiple model training parameters are aggregated to obtain an updated model to be trained. The updated model to be trained is distributed to each federated training terminal. The above steps are repeated until the training process of the model to be trained meets the convergence condition. The model to be trained obtained from the last aggregation is determined as the data analysis model for performing the data analysis task.

[0022] The scheme disclosed herein allows different terminals to be divided into different federated training groups, with different head networks configured for each group. During training, both the head network and the backbone network are trained together. On one hand, this avoids the problem of significant differences among multiple participating terminals. By dividing the terminals into multiple groups and training the backbone network and the corresponding head network within each group using multiple terminals, the convergence speed of learning is improved, and the overall model integrity is enhanced. On the other hand, the data analysis model trained in this way yields more accurate results when performing data analysis tasks.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0025] Figure 1 A flowchart illustrating the federated training method applicable to embodiments of this disclosure is shown schematically.

[0026] Figure 2 This schematically illustrates a flowchart of how, in an exemplary embodiment of this disclosure, when the global training round of federated training meets preset conditions, the updated header network corresponding to each federated training terminal is sent to each federated training terminal.

[0027] Figure 3 This schematically illustrates a flowchart of determining an updated model to be trained based on a fixed backbone network and an updated head network in an exemplary embodiment of this disclosure.

[0028] Figure 4 This schematically illustrates a flowchart of the process of sending updated header networks to each federated training terminal in an exemplary embodiment of this disclosure.

[0029] Figure 5 This schematically illustrates a flowchart of a data analysis process in an exemplary embodiment of the present disclosure, in which target data is analyzed through the backbone network in the data analysis model and the head network corresponding to the federated training group to obtain data analysis results.

[0030] Figure 6 This illustration schematically depicts a data analysis model in an exemplary embodiment of the present disclosure;

[0031] Figure 7 This illustration schematically demonstrates another federated training method in an exemplary embodiment of the present disclosure;

[0032] Figure 8 This schematic diagram illustrates the composition of a federated training device in an exemplary embodiment of the present disclosure.

[0033] Figure 9 This schematic diagram illustrates the composition of a data analysis apparatus according to an exemplary embodiment of the present disclosure;

[0034] Figure 10 The schematic diagram illustrates a structural schematic of a computer system suitable for implementing an electronic device according to exemplary embodiments of the present disclosure. Detailed Implementation

[0035] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., may be employed. In other instances, well-known structures, methods, apparatuses, implementations, materials, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0036] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, or in one or more software-hardened modules, or in different network and / or processor devices and / or microcontroller devices.

[0037] With the rapid development of communication services, it is often necessary to analyze user data in order to provide better services.

[0038] In related technologies, when a data analysis task needs to be performed, user data corresponding to the data analysis task can be collected from the terminal device and uploaded to the data analysis network element. The process of model training in the data analysis network element can be achieved through multiple terminal devices without leaving the local user data.

[0039] However, in related technologies, the terminals participating in model training often vary significantly. For example, from a geographical distribution perspective, some terminals are located in cities, while others are located in rural areas. Significant differences between terminals can lead to slower convergence of federated learning or poor overall model consistency, resulting in poor performance in data analysis.

[0040] According to the federated training method provided in this exemplary embodiment, terminal feature information of multiple federated training terminals can be obtained, and the multiple federated training terminals can be divided into multiple federated training groups based on the terminal feature information; a model to be trained is obtained based on the terminal feature information and the multiple federated training groups; wherein, the model to be trained includes a backbone network and head networks corresponding to the multiple federated training groups; the backbone network and the head networks corresponding to each federated training terminal are distributed to each federated training terminal so that each federated training terminal trains the backbone network and the head networks corresponding to each federated training terminal based on the terminal data of the federated training terminal; model training parameters returned by multiple federated training terminals are received, and the multiple model training parameters are aggregated to obtain an updated model to be trained; the updated model to be trained is distributed to each federated training terminal; the above steps are repeated until the training process of the model to be trained meets the convergence condition, and the model to be trained obtained by the last aggregation is determined as the data analysis model for performing the data analysis task. Figure 1 As shown, the federated training method may include the following steps:

[0041] Step S110: Obtain terminal feature information of multiple federated training terminals, and divide the multiple federated training terminals into multiple federated training groups based on the terminal feature information.

[0042] Step S120: Obtain the model to be trained based on terminal feature information and multiple federated training groups; wherein, the model to be trained includes a backbone network and head networks corresponding to multiple federated training groups.

[0043] Step S130: The backbone network and the head network corresponding to each federated training terminal are distributed to each federated training terminal so that each federated training terminal can train the backbone network and the head network corresponding to each federated training terminal based on the terminal data of the federated training terminal to obtain the model training parameters of each federated training terminal; wherein, the model training parameters include the updated backbone network and the updated head network corresponding to each federated training group.

[0044] Step S140: Receive model training parameters returned by multiple federated training terminals, and aggregate the multiple model training parameters to obtain the updated model to be trained.

[0045] Step S150: Distribute the updated model to be trained to each federated training terminal;

[0046] Step S160: Repeat the above steps until the training process of the model to be trained meets the convergence condition, and determine the model to be trained obtained from the last aggregation as the data analysis model for performing the data analysis task.

[0047] The scheme disclosed herein allows different terminals to be divided into different federated training groups, with different head networks configured for each group. During training, both the head network and the backbone network are trained together. On one hand, this avoids the problem of significant differences among multiple participating terminals. By dividing the terminals into multiple groups and training the backbone network and the corresponding head network within each group using multiple terminals, the convergence speed of learning is improved, and the overall model integrity is enhanced. On the other hand, the data analysis model trained in this way yields more accurate results when performing data analysis tasks.

[0048] The solution disclosed herein can be applied to federated learning scenarios among multiple AI (Artificial Intelligence) network elements in 5G (5th generation mobile networks) networks, such as federated learning among multiple NWDAF (Network Data Analytics Function); the solution disclosed herein can also be applied to federated learning scenarios in wireless networks; the solution disclosed herein can also be applied to scenarios in 6G networks where various AI network elements collaborate on federated training.

[0049] It should be noted that this disclosure does not impose any special limitations on the application scenarios.

[0050] The steps S110 to S160 of the federated training method in this exemplary embodiment will now be described in more detail with reference to the accompanying drawings and embodiments.

[0051] Step S110: Obtain terminal feature information of multiple federated training terminals, and divide the multiple federated training terminals into multiple federated training groups based on the terminal feature information.

[0052] In one example embodiment of this disclosure, terminal characteristic information of multiple federated training terminals can be obtained. Specifically, federated training terminals may include various types of local terminals, such as command-line interface terminals, graphical user interface terminals, remote terminals, and web terminals. It should be noted that this disclosure does not impose any special limitations on the specific type of federated training terminal.

[0053] In one example embodiment of this disclosure, terminal characteristic information of the federated training terminal can be obtained. Specifically, the terminal characteristic information of the federated training terminal can be used to indicate relevant information about the federated training terminal.

[0054] For example, the terminal characteristic information of a federated training terminal may include location information, busy time information, idle time information, data volume information, data type information, local computing power information, etc.

[0055] It should be noted that this disclosure does not impose any special restrictions on the specific types of terminal characteristic information of the federated training terminal.

[0056] In one exemplary embodiment of this disclosure, after obtaining the terminal feature information of multiple federated training terminals through the above steps, the multiple federated training terminals can be divided into multiple federated training groups based on the terminal feature information. Specifically, the multiple federated training terminals can be divided into multiple federated training groups based on one type of terminal feature information, or multiple types of terminal feature information can be used to divide the multiple federated training terminals into multiple federated training groups.

[0057] For example, terminal feature information can be location information, meaning that terminal feature information can be used to indicate the location of the federated training terminal. Based on this location information, multiple federated training terminals can be divided into multiple federated training groups. For instance, federated training terminals belonging to the same region can be grouped into the same federated training group.

[0058] It should be noted that this disclosure does not impose any special restrictions on the specific method of dividing multiple federated training terminals into multiple federated training groups based on terminal characteristic information.

[0059] Step S120: Obtain the model to be trained based on terminal feature information and multiple federated training groups;

[0060] In one example embodiment of this disclosure, after dividing multiple federated training terminals into multiple federated training groups through the above steps, the model to be trained can be obtained based on the terminal feature information and the multiple federated training groups. Specifically, the backbone network refers to the network part shared by each federated training terminal during the training process, and the head network corresponding to each federated training group refers to the network part used independently by each federated training terminal during the training process.

[0061] In one example embodiment of this disclosure, after determining multiple federated training groups, a model to be trained can be generated based on terminal feature information and multiple federated training groups. Specifically, the generation method of the head network corresponding to each federated training group in the model to be trained is related to the terminal feature information of each federated training group and the federated training terminal in each federated training group.

[0062] Furthermore, the number of head networks, the specific topology of each head network, the topology of the backbone network, and the federated learning hyperparameters of the model to be trained can be determined based on terminal feature information and multiple federated training groups. Specifically, each federated training group corresponds to one head network, and the number of head networks is consistent with the number of federated training groups. The specific topology of the head network includes the number of network layers, the connection method between network layers, and the number of parameters, which can be determined based on the features of the federated training groups. The topology of the backbone network includes the number of network layers, the connection method between layers, and the number of parameters. Federated learning hyperparameters include the learning rate, data batch size, dropout rate, and number of local training epochs, etc.

[0063] For example, if the data in a federated training group is highly time-dependent, a time-sensitive model structure can be considered; if the federated training terminals included in the federated training group have low computing power or small data volume, the size of the head model can be reduced.

[0064] Step S130: The backbone network and the head network corresponding to each federated training terminal are sent to each federated training terminal so that each federated training terminal can train the backbone network and the head network corresponding to each federated training terminal based on the terminal data of the federated training terminal, and obtain the model training parameters of each federated training terminal.

[0065] In one example embodiment of this disclosure, after obtaining the model to be trained through the above steps, the backbone network and the head network corresponding to each federated training terminal can be distributed to each federated training terminal. Here, the head network corresponding to a federated training terminal refers to a federated training group corresponding to that federated training terminal, which in turn corresponds to a head network. The head network corresponding to that federated training group can be used as the head network corresponding to the federated training terminal.

[0066] Specifically, when determining whether to distribute the backbone network and the corresponding head network to each federated training terminal, or only distribute the corresponding head network to each federated training terminal, the determination can be made based on the fixed rounds of the backbone network.

[0067] It should be noted that this disclosure does not specify the method for determining whether to distribute the backbone network and the corresponding header network to each federated training terminal, or only distribute the corresponding header network to each federated training terminal.

[0068] In one exemplary embodiment of this disclosure, after the backbone network and the corresponding head networks of each federated training terminal are distributed to each federated training terminal through the above steps, each federated training terminal trains the backbone network and the corresponding head networks based on its terminal data to obtain the model training parameters for each federated training terminal. Specifically, the federated training terminal trains the backbone network and the corresponding head networks based on its terminal data. At this time, the backbone network and the corresponding head networks can be trained to obtain the model training parameters for the federated training terminal (including the model training parameters of the backbone network and the model training parameters of the corresponding head networks).

[0069] Furthermore, a federated training terminal can be selected from multiple candidate federated training terminals, and the backbone network and the corresponding head network of each federated training terminal can be distributed to each federated training terminal.

[0070] Step S140: Receive model training parameters returned by multiple federated training terminals, and aggregate the multiple model training parameters to obtain the updated model to be trained.

[0071] In one example embodiment of this disclosure, after obtaining the model training parameters of each federated training terminal through the above steps, model training parameters returned by multiple federated training terminals can be received, and the multiple model training parameters can be aggregated to obtain an updated model to be trained. The updated model to be trained may include an updated backbone network and updated head networks corresponding to each federated training group. Specifically, aggregating multiple model training parameters refers to performing calculations on these model training parameters in a preset manner to generate new, updated model training parameters.

[0072] For example, aggregating training parameters of multiple models can be done in several ways, such as simple averaging, weighted averaging, and aggregation functions.

[0073] Step S150: Distribute the updated model to be trained to each federated training terminal;

[0074] In one example embodiment of this disclosure, after obtaining the updated model to be trained through the above steps, the updated model to be trained can be distributed to each federated training terminal. Specifically, the updated model to be trained can be distributed to each federated training terminal so that each federated training terminal can train the updated backbone network and the updated head network corresponding to each federated training terminal based on the terminal data of each federated training terminal.

[0075] It should be noted that this disclosure does not impose any special restrictions on the specific method of distributing the updated models to be trained to each federated training terminal.

[0076] Step S160: Repeat the above steps until the training process of the model to be trained meets the convergence condition, and determine the model to be trained obtained from the last aggregation as the data analysis model for performing the data analysis task.

[0077] In one example embodiment of this disclosure, the above steps can be repeated, namely, receiving model training parameters returned by multiple federated training terminals, aggregating the multiple model training parameters to obtain an updated model to be trained (including the updated backbone network and the updated head network corresponding to each federated training group), and continuing to distribute the backbone network and the head network corresponding to each federated training terminal to each federated training terminal. After each federated training terminal obtains the backbone network and the head network corresponding to each federated training terminal, each federated training terminal trains the backbone network and the head network corresponding to each federated training terminal according to the terminal data of the federated training terminal to obtain the model training parameters of each federated training terminal. Each federated training terminal then uploads the model training parameters of the federated training terminal. After receiving the model training parameters returned by multiple federated training terminals, the aggregation is continued to obtain an updated model to be trained, and so on, until the training process of the model to be trained meets the convergence condition. At this time, a data analysis model for performing data analysis tasks can be obtained.

[0078] In one exemplary embodiment of this disclosure, the convergence condition refers to the condition under which the iterative process of the model to be trained reaches a certain point, such that the performance of the model tends to stabilize or reach the optimal solution, and the iteration stops, thus obtaining the data analysis model. For example, the convergence condition could be stopping training when the training error or validation error starts to increase, or stopping training when the validation set error no longer decreases.

[0079] In one example embodiment of this disclosure, the model training parameters of the backbone network and the model training parameters of the head networks corresponding to each federated training group can be aggregated to obtain an updated model to be trained. The model training parameters include the model training parameters of the backbone network and the model training parameters of the head networks corresponding to each federated training group. Specifically, when aggregating the model training parameters of the head networks corresponding to the federated training groups, it is done based on the model training parameters of the head networks returned by multiple federated training terminals within the same federated training group.

[0080] Specifically, when aggregating the model training parameters of the backbone network, the following formula can be used:

[0081]

[0082] Where w represents the backbone network obtained after aggregation, K represents the number of federated training terminals participating in this round of training, and n represents the total data volume of the K federated training terminals.k Let be the amount of data for the k-th federated training terminal.

[0083] Specifically, when aggregating the model training parameters of the head network, the following formula can be used:

[0084]

[0085] Among them, h m This is the head network obtained after aggregating the data from the m-th federated training group, where M is the number of federated training terminals participating in this round of training and belonging to the m-th federated training group, and n is the total amount of data from the M federated training terminals. k Let be the amount of data for the k-th federated training terminal.

[0086] It should be noted that this disclosure does not impose any special restrictions on the specific method of aggregating the model training parameters of the backbone network and the model training parameters of the head networks corresponding to each federated training group to obtain the updated model to be trained.

[0087] In one example embodiment of this disclosure, the global training epochs of federated training can be taken. When the global training epochs of federated training meet preset conditions, the updated head network corresponding to each federated training terminal is sent to each federated training terminal. This allows each federated training terminal to train its corresponding head network based on its terminal data, thereby obtaining the model training parameters for each federated training terminal. (Refer to...) Figure 2 As shown, when the global training rounds of federated training meet the preset conditions, the updated header network corresponding to each federated training terminal is sent to each federated training terminal, which may include the following steps S210 to S220:

[0088] Step S210: Obtain the global training rounds for federated training;

[0089] In one example embodiment of this disclosure, the global training rounds of federated training can be obtained. Specifically, the global training rounds of federated training refer to the total number of training rounds from the start of training to the current training, that is, the number of times communication and model training parameters are transmitted between the federated training terminal and the federated training terminal.

[0090] Step S220: When the global training rounds of federated training meet the preset conditions, the updated head network corresponding to each federated training terminal is sent to each federated training terminal so that each federated training terminal can train the head network corresponding to each federated training terminal based on the terminal data of the federated training terminal, thereby obtaining the model training parameters of each federated training terminal.

[0091] In one example embodiment of this disclosure, when the global training rounds of federated training meet preset conditions, the updated head network and fixed backbone network corresponding to each federated training terminal are distributed to each federated training terminal. Specifically, it can be determined whether the global training rounds of federated training meet preset conditions. If the global training rounds do not meet the preset conditions, it means that the backbone network has not yet been trained. When training is performed on the federated training terminal, both the backbone network and the head network need to be trained simultaneously, that is, the updated backbone network and the corresponding head network of each federated training terminal need to be distributed to each federated training terminal. If the global training rounds meet the preset conditions, it means that the backbone network has been trained. When training is performed on the federated training terminal, only the head network needs to be trained, that is, only the updated head network corresponding to each federated training terminal needs to be distributed to each federated training terminal. This allows each federated training terminal to train its corresponding head network based on its terminal data, obtaining the model training parameters for each federated training terminal. The trained model training parameters include the fixed backbone network and the updated head network corresponding to each federated training terminal.

[0092] Specifically, if the above steps involve distributing the backbone network and the corresponding head network of the federated training terminal to the federated training terminal, then the federated training terminal trains the backbone network and the corresponding head network based on its terminal data. In this case, the backbone network and the corresponding head network of the federated training terminal can be trained to obtain the model training parameters of the federated training terminal (including the updated model training parameters of the backbone network and the corresponding head network of the federated training terminal). If the above steps involve distributing the head networks corresponding to each federated training terminal to the federated training terminal, then the model training parameters in the backbone network are fixed, and only the head network corresponding to the federated training terminal is trained to obtain the model training parameters of the federated training terminal (including the updated model training parameters of the corresponding head network of the federated training terminal and the fixed backbone network).

[0093] Furthermore, when the backbone network is not fixed, a larger number of federated training terminals can be selected; after the backbone network is fixed, a smaller number of federated training terminals can be selected.

[0094] Through the above steps S210 to S220, the global training rounds of federated training can be obtained. When the global training rounds of federated training meet the preset conditions, the updated head network corresponding to each federated training terminal is sent to each federated training terminal so that each federated training terminal can train the head network corresponding to each federated training terminal according to the terminal data of the federated training terminal and obtain the model training parameters of each federated training terminal.

[0095] In one example embodiment of this disclosure, the model training parameters of the head networks corresponding to each federated training group can be aggregated to obtain an updated head network. The updated model to be trained is then determined based on the fixed backbone network and the updated head network. (Refer to...) Figure 3 As shown, determining the updated model to be trained based on the fixed backbone network and the updated head network may include the following steps S310 to S320:

[0096] Step S310: Aggregate the model training parameters of the head network corresponding to each federated training group to obtain the updated head network.

[0097] In one example embodiment of this disclosure, after obtaining the model training parameters of each federated training terminal through the above steps, model training parameters returned by multiple federated training terminals can be received. The updated model to be trained may include a fixed backbone network and updated head networks corresponding to each federated training group. Specifically, the updated head network can be obtained by aggregating only the model training parameters of the head networks corresponding to each federated training group. Aggregating multiple model training parameters refers to performing calculations on these model training parameters in a preset manner to generate new, updated model training parameters.

[0098] For example, aggregating training parameters of multiple models can be done in several ways, such as simple averaging, weighted averaging, and aggregation functions.

[0099] It should be noted that this disclosure does not impose any special restrictions on the specific method of aggregating the model training parameters of the head networks corresponding to each federated training group to obtain the updated head network.

[0100] Step S320: Determine the updated model to be trained based on the fixed backbone network and the updated head network.

[0101] In one exemplary embodiment of this disclosure, after obtaining the updated head network through the above steps, the updated model to be trained can be determined based on the fixed backbone network and the updated head network. Specifically, since the backbone network is already fixed, that is, the model training parameters of the backbone network are fixed, the backbone network and the updated head network can be directly merged to obtain the updated model to be trained.

[0102] It should be noted that this disclosure does not impose any special limitations on the specific method of determining the updated model to be trained based on the fixed backbone network and the updated head network.

[0103] Through the above steps S310 to S320, the model training parameters of the head network corresponding to each federated training group can be aggregated to obtain the updated head network. The updated model to be trained is determined based on the fixed backbone network and the updated head network.

[0104] In one example embodiment of this disclosure, a fixed number of epochs for the backbone network corresponding to the model to be trained can be determined. When the total number of global training epochs in the federated training is greater than the fixed number of epochs for the backbone network, the updated head network corresponding to each federated training terminal is distributed to each federated training terminal. (Refer to...) Figure 4 As shown, distributing the updated header network corresponding to each federated training terminal to each federated training terminal may include the following steps S410 to S420:

[0105] Step S410: Determine the fixed number of rounds of the backbone network corresponding to the model to be trained;

[0106] In one example embodiment of this disclosure, the number of fixed epochs for the backbone network corresponding to the model to be trained can be determined. Specifically, the fixed epochs for the backbone network means that after the fixed epochs for the backbone network are performed in the federated training, updates to the backbone network are stopped (i.e., the backbone network is fixed). During the federated training process after the fixed epochs for the backbone network, only the head network is updated.

[0107] It should be noted that this disclosure does not impose any special restrictions on the specific method for determining the fixed number of rounds of the backbone network corresponding to the model to be trained, and can be adjusted according to the business scenario.

[0108] Step S420: When the global training rounds of federated training are greater than the fixed rounds of the backbone network, the updated head network corresponding to each federated training terminal is sent to each federated training terminal.

[0109] In one example embodiment of this disclosure, the global training epochs can be compared with the fixed epochs of the backbone network to obtain the relationship between the global training epochs and the fixed epochs of the backbone network in the federated training. When the global training epochs of the federated training are greater than the fixed epochs of the backbone network, the updated head network corresponding to each federated training terminal is distributed to each federated training terminal. Specifically, when the global training epochs are less than or equal to the fixed epochs of the backbone network, it indicates that the backbone network has not yet been trained. When training is performed on the federated training terminal, both the backbone network and the head network need to be trained simultaneously, that is, the updated backbone network and the corresponding head network for each federated training terminal need to be distributed to each federated training terminal. When the global training epochs are greater than the fixed epochs of the backbone network, it indicates that the backbone network has been trained. When training is performed on the federated training terminal, only the head network needs to be trained, that is, the updated head network corresponding to each federated training terminal needs to be distributed to each federated training terminal.

[0110] In one example embodiment of this disclosure, the above steps can be repeated: receiving model training parameters returned by multiple federated training terminals, aggregating the multiple model training parameters to obtain an updated model to be trained (including the updated head network corresponding to each federated training group and the fixed backbone network), then distributing the updated head network corresponding to each federated training terminal to each federated training terminal, and after each federated training terminal obtains its corresponding head network, it trains its corresponding head network according to its terminal data to obtain model training parameters for each federated training terminal, and then uploads its model training parameters. After receiving the model training parameters returned by multiple federated training terminals, the aggregation is continued to obtain an updated model to be trained, and so on, until the training process of the model to be trained meets the convergence condition. At this point, a data analysis model for performing data analysis tasks can be obtained.

[0111] Furthermore, when the global training rounds corresponding to the federated training terminal are greater than the fixed rounds of the backbone network, there are two scenarios. One is that the federated training terminal has already received the fixed backbone network, i.e., the updated backbone network has been sent to the federated training terminal. In this case, the head network corresponding to the federated training terminal can be trained based on the terminal data of the federated training terminal, and only the head network corresponding to the federated training terminal needs to be sent to the federated training terminal. The other is that the federated training terminal has not received the fixed backbone network, i.e., the fixed backbone network has not been sent to the federated training terminal, but the backbone network has already been trained. In this case, the head network corresponding to the federated training terminal can be trained based on the terminal data of the federated training terminal, i.e., both the fixed backbone network and the head network corresponding to the federated training terminal need to be sent to the federated training terminal.

[0112] Through the above steps S410 to S420, the fixed number of backbone network rounds corresponding to the model to be trained can be determined. When the global training rounds of federated training are greater than the fixed number of backbone network rounds, the updated head network corresponding to each federated training terminal is sent to each federated training terminal.

[0113] In one example embodiment of this disclosure, a data analysis request sent by a terminal device is received, the federated training group to which the terminal device belongs is determined, the target data is input into the data analysis model, and the target data is analyzed through the backbone network in the data analysis model and the head network corresponding to the federated training group to obtain the data analysis results. (Refer to...) Figure 5 As shown, the data analysis results are obtained by analyzing the target data through the backbone network in the data analysis model and the head network corresponding to the federated training group. This may include the following steps S510 to S520:

[0114] Step S510: Receive a data analysis request sent by the terminal device and determine the federated training group to which the terminal device belongs; wherein, the data analysis request includes the target data to be analyzed;

[0115] Step S520: Input the target data into the data analysis model, and analyze the target data through the backbone network in the data analysis model and the head network corresponding to the federated training group to obtain the data analysis results.

[0116] In one example embodiment of this disclosure, a data analysis request sent by a terminal device can be received. Specifically, the data analysis request includes target data to be analyzed, and the data analysis request is used to analyze the target data.

[0117] Furthermore, data analysis tasks can be sent by other network elements or servers and received by the data analysis network element. For example, data analysis tasks can be sent through application layer network elements and received by the data analysis network element; or, data analysis tasks can be sent through session management network elements and received by the data analysis network element; or, data analysis tasks can be sent through network data repository network elements and received by the data analysis network element.

[0118] It should be noted that this disclosure does not impose any special restrictions on the subject sending the data analysis task or the specific type of the data analysis task.

[0119] In one example embodiment of this disclosure, the federated training group to which the terminal device belongs can be determined. Specifically, terminal feature information of the terminal device can be obtained, and the federated training group to which the terminal device belongs can be determined based on the terminal feature information.

[0120] In one exemplary embodiment of this disclosure, after obtaining the target data and the federated training group to which the terminal device belongs through the above steps, the target data can be input into the data analysis model. The target data is then analyzed by the backbone network in the data analysis model and the head network corresponding to the federated training group to obtain the data analysis results. The data analysis model is obtained through the federated training method as described in any of the above embodiments. Specifically, the target data first passes through the backbone network of the data analysis model, and then through the head network corresponding to the federated training group to obtain the data analysis results.

[0121] Through the above steps S510 to S520, a data analysis request sent by the terminal device can be received, the federated training group to which the terminal device belongs can be determined, the target data can be input into the data analysis model, and the target data can be analyzed through the backbone network in the data analysis model and the head network corresponding to the federated training group to obtain the data analysis results.

[0122] In one exemplary embodiment of this disclosure, such as Figure 6The diagram illustrates a data analysis model. After obtaining the target data and the federated training group to which the terminal device belongs, the target data can be input into the encoding unit for encoding. The encoded data is then input into the backbone network. The federated training group to which the terminal device belongs is determined from among multiple head networks. The output of the backbone network is then input into the head network corresponding to the federated training group, and the output of the head network is input into the decoding unit. The decoded result is the data analysis result.

[0123] In one exemplary embodiment of this disclosure, such as Figure 7 As shown, another federated training method includes the following steps S701 to S708:

[0124] Step S701: The federated training terminal can send terminal feature information to the data analysis node;

[0125] Step S702: In the data analysis node, multiple federated training terminals are divided into multiple federated training groups according to terminal feature information, and the model to be trained is obtained according to the terminal feature information and multiple federated training groups.

[0126] Step S703: The data analysis node sends the generated model to be trained to the model aggregation node;

[0127] Step S704: The model aggregation node selects the federated training terminal participating in this training.

[0128] Step S705: The model aggregation node distributes the model to be trained to the federated training terminal.

[0129] Step S706: The federated training terminal trains the backbone network and the head network corresponding to each federated training terminal (when the backbone network is not fixed and the global training rounds of the federated training are less than or equal to the fixed rounds of the backbone network) / the head network corresponding to each federated training terminal (when the backbone network is fixed and the global training rounds of the federated training are greater than the fixed rounds of the backbone network) based on the terminal data of the federated training terminal, and obtains the model training parameters of each federated training terminal.

[0130] Step S707: The federated training terminal uploads the training parameters of the trained model to the model aggregation node.

[0131] Step S708: The model aggregation node aggregates the training parameters of multiple models to obtain the updated model to be trained.

[0132] In one embodiment of the federated training method or data analysis method provided by this disclosure, terminal feature information of multiple federated training terminals can be obtained. Based on the terminal feature information, the multiple federated training terminals are divided into multiple federated training groups. A model to be trained is obtained based on the terminal feature information and the multiple federated training groups. The backbone network and the corresponding head network of each federated training terminal are distributed to each federated training terminal so that each federated training terminal trains the backbone network and the corresponding head network based on the terminal data of the federated training terminal, obtaining the model training parameters of each federated training terminal. The model training parameters returned by multiple federated training terminals are received, and the multiple model training parameters are aggregated to obtain an updated model to be trained. The updated model to be trained is distributed to each federated training terminal. The above steps are repeated until the training process of the model to be trained meets the convergence condition. The model to be trained obtained from the last aggregation is determined as the data analysis model for performing the data analysis task.

[0133] The scheme disclosed herein allows different terminals to be divided into different federated training groups, with different head networks configured for each group. During training, both the head network and the backbone network are trained together. On one hand, this avoids the problem of significant differences among multiple participating terminals. By dividing the terminals into multiple groups and training the backbone network and the corresponding head network within each group using multiple terminals, the convergence speed of learning is improved, and the overall model integrity is enhanced. On the other hand, the data analysis model trained in this way yields more accurate results when performing data analysis tasks.

[0134] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0135] Furthermore, in an exemplary embodiment of this disclosure, a federated training apparatus is also provided. (Refer to...) Figure 8 As shown, a federated training device 800 includes: a federated training group division module 810, a model generation module 820, a network distribution module 830, a model aggregation module 840, a distribution module 850, and a model training module 860.

[0136] The system comprises the following modules: a federated training group partitioning module, used to acquire terminal feature information from multiple federated training terminals and divide them into multiple federated training groups based on this information; a model generation module, used to generate a model to be trained based on the terminal feature information and the multiple federated training groups, wherein the model to be trained includes a backbone network and head networks corresponding to the multiple federated training groups; a network distribution module, used to distribute the backbone network and the head networks corresponding to each federated training terminal to each federated training terminal, so that each federated training terminal can train the backbone network and the head networks corresponding to each federated training terminal based on the terminal data of the federated training terminal, thereby obtaining the model training parameters for each federated training terminal; a model aggregation module, used to receive the model training parameters returned by multiple federated training terminals and aggregate the multiple model training parameters to obtain an updated model to be trained; a distribution module, used to distribute the updated model to be trained to each federated training terminal; and a model training module, used to repeat the above steps until the training process of the model to be trained meets the convergence condition, and the model to be trained obtained from the last aggregation is determined as the data analysis model for performing the data analysis task.

[0137] In an exemplary embodiment of this disclosure, based on the aforementioned scheme, the model training parameters include the model training parameters of the backbone network and the model training parameters of the head network corresponding to each federated training group. The updated model to be trained is obtained by aggregating multiple model training parameters. The apparatus further includes: a first aggregation unit, used to aggregate the model training parameters of the backbone network and the model training parameters of the head network corresponding to each federated training group to obtain the updated model to be trained.

[0138] In an exemplary embodiment of this disclosure, based on the foregoing scheme, before receiving model training parameters returned by multiple federated training terminals, the apparatus further includes: a global training round acquisition unit, used to acquire the global training rounds of the federated training; and a distribution unit, used to distribute the updated head network corresponding to each federated training terminal to each federated training terminal when the global training rounds of the federated training meet preset conditions, so that each federated training terminal can train the head network corresponding to each federated training terminal according to the terminal data of the federated training terminal to obtain the model training parameters of each federated training terminal.

[0139] In an exemplary embodiment of this disclosure, based on the aforementioned scheme, multiple model training parameters are aggregated to obtain an updated model to be trained. The apparatus further includes: a second aggregation unit, used to aggregate the model training parameters of the head network corresponding to each federated training group to obtain an updated head network; and a model to be trained update unit, used to determine the updated model to be trained based on the fixed backbone network and the updated head network.

[0140] In one exemplary embodiment of this disclosure, based on the aforementioned scheme, when the global training rounds of federated training meet preset conditions, the updated head network corresponding to each federated training terminal is sent to each federated training terminal. The device further includes: a backbone network fixed rounds determination unit, used to determine the fixed rounds of the backbone network corresponding to the model to be trained; and a head network sending unit, used to send the updated head network corresponding to each federated training terminal to each federated training terminal when the global training rounds of federated training are greater than the fixed rounds of the backbone network.

[0141] Since the functional modules of the federated training device in the example embodiments of this disclosure correspond to the steps of the example embodiments of the federated training method described above, for details not disclosed in the device embodiments of this disclosure, please refer to the embodiments of the federated training method described above.

[0142] Furthermore, in an exemplary embodiment of this disclosure, a data analysis apparatus is also provided. (Refer to...) Figure 9 As shown, a data analysis device 900 includes an analysis request receiving module 910 and a data analysis module 920.

[0143] The analysis request receiving module is used to receive data analysis requests sent by terminal devices and determine the federated training group to which the terminal device belongs. The data analysis request includes the target data to be analyzed. The data analysis module is used to input the target data into the data analysis model and analyze the target data through the backbone network in the data analysis model and the head network corresponding to the federated training group to obtain the data analysis results. The data analysis model is trained by any of the federated training methods mentioned above.

[0144] Since the functional modules of the data analysis device in the example embodiments of this disclosure correspond to the steps of the example embodiments of the data analysis method described above, for details not disclosed in the device embodiments of this disclosure, please refer to the embodiments of the data analysis method described above.

[0145] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0146] Furthermore, in an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described federated training method and data analysis method is also provided.

[0147] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be embodied in the following forms: a completely hardware embodiment, a completely software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0148] The following reference Figure 10 To describe an electronic device 1000 according to such an embodiment of the present disclosure. Figure 10 The electronic device 1000 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0149] like Figure 10 As shown, the electronic device 1000 is manifested in the form of a general-purpose computing device. The components of the electronic device 1000 may include, but are not limited to: at least one processing unit 1010, at least one storage unit 1020, a bus 1030 connecting different system components (including storage unit 1020 and processing unit 1010), and a display unit 1040.

[0150] The storage unit stores program code, which can be executed by the processing unit 1010 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 1010 can perform actions such as... Figure 1 Step S110 shows obtaining terminal feature information of multiple federated training terminals and dividing the multiple federated training terminals into multiple federated training groups based on the terminal feature information; Step S120 shows obtaining the model to be trained based on the terminal feature information and the multiple federated training groups; wherein, the model to be trained includes a backbone network and head networks corresponding to the multiple federated training groups; Step S130 shows distributing the backbone network and the head networks corresponding to each federated training terminal to each federated training terminal, so that each federated training terminal can train the backbone network and the head networks corresponding to each federated training terminal based on the terminal data of the federated training terminal. The process involves several steps: Step S140: Receiving model training parameters from multiple federated training terminals and aggregating these parameters to obtain an updated model to be trained; Step S150: Distributing the updated model to be trained to each federated training terminal; Step S160: Repeating the above steps until the training process of the model to be trained meets the convergence condition, and determining the model to be trained obtained from the last aggregation as the data analysis model for performing the data analysis task.

[0151] Alternatively, the processing unit 1010 can perform, as follows: Figure 5 In step S510, a data analysis request is received from the terminal device, and the federated training group to which the terminal device belongs is determined; wherein, the data analysis request includes the target data to be analyzed; in step S520, the target data is input into the data analysis model, and the target data is analyzed through the backbone network in the data analysis model and the head network corresponding to the federated training group to obtain the data analysis result; wherein, the data analysis model is obtained through the federated training method as described in any of the above embodiments.

[0152] For example, electronic devices can achieve such Figure 1 or Figure 5 The steps shown.

[0153] Storage unit 1020 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 1021 and / or a cache memory unit 1022, and may further include a read-only memory unit (ROM) 1023.

[0154] Storage unit 1020 may also include a program / utility 1024 having a set (at least one) program module 1025, such program module 1025 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0155] Bus 1030 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the multiple bus structures.

[0156] Electronic device 1000 can also communicate with one or more external devices 1070 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1000, and / or any device that enables electronic device 1000 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1050. Furthermore, electronic device 1000 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1060. As shown, network adapter 1060 communicates with other modules of electronic device 1000 via bus 1030. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0157] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0158] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product including program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0159] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0160] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0161] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0162] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0163] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A federated training method, characterized in that, The method comprises: obtaining terminal characteristic information of a plurality of federal training terminals, and dividing the plurality of federal training terminals into a plurality of federal training groups according to the terminal characteristic information; the terminal characteristic information comprises location, busy time, idle time, data volume, data type and local computing power; obtaining a to-be-trained model according to the terminal characteristic information and the plurality of federal training groups; wherein the to-be-trained model comprises a backbone network and a plurality of head networks corresponding to the plurality of federal training groups; downloading the backbone network and the head network corresponding to each of the federal training terminals to each of the federal training terminals, so that each of the federal training terminals trains the backbone network and the head network corresponding to each of the federal training terminals according to terminal data of the federal training terminal to obtain model training parameters of each of the federal training terminals; the head network corresponding to each of the federal training terminals is the same as the head network corresponding to the federal training group to which the federal training terminal belongs; receiving model training parameters returned by the plurality of federal training terminals, and aggregating the plurality of model training parameters to obtain an updated to-be-trained model; downloading the updated to-be-trained model to each of the federal training terminals; repeating the above steps until the training process of the to-be-trained model meets a convergence condition, and determining the to-be-trained model obtained by the last aggregation as a data analysis model for executing a data analysis task.

2. The method of claim 1, wherein, The model training parameters comprise model training parameters of the backbone network and model training parameters of the head network corresponding to each of the federal training groups, and the aggregation of the plurality of model training parameters to obtain the updated to-be-trained model comprises: aggregating the model training parameters of the backbone network and the model training parameters of the head network corresponding to each of the federal training groups to obtain the updated to-be-trained model.

3. The method of claim 1, wherein, Before the receiving of the model training parameters returned by the plurality of federal training terminals, the method further comprises: obtaining a global training round of the federal training; when the global training round of the federal training meets a preset condition, downloading updated head networks corresponding to each of the federal training terminals to each of the federal training terminals, so that each of the federal training terminals trains the head network corresponding to each of the federal training terminals according to terminal data of the federal training terminal to obtain model training parameters of each of the federal training terminals.

4. The method of claim 3, wherein, The aggregation of the plurality of model training parameters to obtain the updated to-be-trained model comprises: aggregating the model training parameters of the head network corresponding to each of the federal training groups to obtain an updated head network; determining the updated to-be-trained model according to the fixed backbone network and the updated head network.

5. The method of claim 3, wherein, The downloading of the updated head networks corresponding to each of the federal training terminals to each of the federal training terminals when the global training round of the federal training meets the preset condition comprises: determining a fixed round of the backbone network corresponding to the to-be-trained model; when the global training round of the federal training is greater than the fixed round of the backbone network, downloading updated head networks corresponding to each of the federal training terminals to each of the federal training terminals.

6. A data analysis method characterized by, The method comprises: receive a data analysis request sent by a terminal device, and determine a federated training group to which the terminal device belongs; the data analysis request comprises target data to be analyzed; input the target data into the data analysis model, and analyze the target data by using a backbone network in the data analysis model and a head network corresponding to the federated training group to obtain a data analysis result; wherein the data analysis model is trained by the federated training method in any one of claims 1-5.

7. A federal training device characterized by, The apparatus comprises: a federated training group division module configured to obtain terminal feature information of a plurality of federated training terminals, and divide the plurality of federated training terminals into a plurality of federated training groups according to the terminal feature information; the terminal feature information comprises location, busy time, idle time, data volume, data type, and local computing power; a to-be-trained model generation module configured to obtain a to-be-trained model according to the terminal feature information and the plurality of federated training groups; the to-be-trained model comprises a backbone network and a plurality of head networks corresponding to the plurality of federated training groups; a network delivery module configured to deliver the backbone network and the head network corresponding to each federated training terminal to each federated training terminal, so that each federated training terminal trains the backbone network and the head network corresponding to each federated training terminal according to terminal data of the federated training terminal, to obtain model training parameters of each federated training terminal; the head network corresponding to each federated training terminal is the same as the head network corresponding to the federated training group to which the federated training terminal belongs; a model aggregation module configured to receive model training parameters returned by the plurality of federated training terminals, and aggregate the plurality of model training parameters to obtain an updated to-be-trained model; a delivery module configured to deliver the updated to-be-trained model to each federated training terminal; a model training module configured to repeat the above steps until a training process of the to-be-trained model meets a convergence condition, and determine the to-be-trained model obtained by the last aggregation as a data analysis model for performing a data analysis task.

8. A data analytics model analysis apparatus, characterized by, The apparatus comprises: an analysis request receiving module configured to receive a data analysis request sent by a terminal device, and determine a federated training group to which the terminal device belongs; the data analysis request comprises target data to be analyzed; a data analysis module configured to input the target data into the data analysis model, and analyze the target data by using a backbone network in the data analysis model and a head network corresponding to the federated training group to obtain a data analysis result; the data analysis model is trained by the federated training method in any one of claims 1-5.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements the method of any one of claims 1-6.

10. An electronic device, comprising: comprises: one or more processors; and a memory for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-6.

Images