A collaborative training method based on multi-data alliance

By calculating link quality and determining training paths in multi-data alliances, collaborative training and personalized adjustments, the problem of low accuracy of collaborative training of data alliance nodes without central servers is solved, and the accuracy and stability of the local model are improved .

CN118798395BActive Publication Date: 2024-12-03CENT SOUTH UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411278271.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-12
Publication Date
2024-12-03
Estimated Expiration
2044-09-12

AI Technical Summary

Technical Problem

Without a centralized advanced server for data aggregation, it is difficult to achieve collaborative training between multiple data alliance nodes, resulting in low accuracy of the local model and cannot be applied to actual needs.

Method used

Through the information calculation alliance node, the link quality of each data alliance node is calculated and the collaborative training path is determined. The task initiator alliance node collaborates on its original local model based on the collaborative training path and the local model of the data provider alliance node. A local composite data set is constructed and a comprehensive evaluation score is calculated. If the score is less than the threshold, personalized training is performed to improve model accuracy.

Benefits of technology

It realizes collaborative training of multiple data alliances without a central server, improves the accuracy of the local model in the data alliance node, and prevents local model from being overfitted, effectively improving the accuracy of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118798395B_ABST
    Figure CN118798395B_ABST
Patent Text Reader

Abstract

The present application is applicable to the field of federated learning technology, and provides a collaborative training method based on a multi-data alliance, the collaborative training method comprising the task initiator alliance node sending a collaborative training request; after receiving the collaborative training request, the information calculation party alliance node collects the communication information of each data alliance node, calculates the link quality, and determines the collaborative training path; the task initiator alliance node performs collaborative training on its original local model, constructs a local composite data set, and calculates the evaluation scores of the collaboratively trained local model and the original local model; if the evaluation score is less than a preset score threshold, the collaboratively trained local model is personalized trained; otherwise, the collaboratively trained local model is used as the final local model. The present application can realize collaborative training between multiple data alliance nodes without a centralized advanced server for data aggregation, and improve the accuracy of the local model in the data alliance node.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of federated learning technology, and in particular, relates to a collaborative training method based on a multi-data alliance. Background Art

[0002] In today's data-driven market, data providers often form alliances to enhance data sharing and analysis capabilities. For example, some companies that mainly provide data for internal business processes and analysis formed an alliance node. At the same time, companies that focus on providing data services to consumers at home also formed an alliance node. The two data alliance nodes operate independently, using federated learning algorithms (such as FedAvg) to process and learn data internally to optimize their respective services and products.

[0003] However, a key challenge is how to achieve collaborative training among multiple data union nodes without a centralized high-level server for data aggregation. In this case, conventional federated learning algorithms cannot be directly applied, resulting in low accuracy of local models of data union nodes and inability to meet actual needs. Summary of the invention

[0004] The present application provides a collaborative training method based on multiple data alliances, which can realize collaborative training between multiple data alliance nodes without a centralized advanced server for data aggregation, thereby improving the accuracy of local models in the data alliance.

[0005] The present application provides a collaborative training method based on a multi-data alliance, wherein the multi-data alliance includes a task initiator alliance node, a data provider alliance node, and an information computing party alliance node, wherein the data provider alliance node includes at least one data alliance node, and the distance between the information computing party alliance node and each data alliance node in the multi-data alliance is less than a preset distance threshold, and the task initiator alliance node and the information computing party alliance node are two different data alliance nodes in the multi-data alliance; the collaborative training method includes:

[0006] The task initiator alliance node sends a collaborative training request to the information computing party alliance node;

[0007] After receiving the collaborative training request, the information computing party alliance node collects the communication information of each data alliance node in the multi-data alliance, calculates the link quality corresponding to each data alliance node according to the communication information, determines the collaborative training path according to the link quality, and sends the collaborative training path to the task initiator alliance node; the communication information includes the arrival rate used to describe the average rate at which each data alliance node receives requests, the service rate used to describe the average rate at which each data alliance node processes requests, and the number of service windows used to process requests in each data alliance node. The collaborative training path indicates the order in which the data alliance nodes in the data provider alliance node perform collaborative training;

[0008] The task initiator alliance node performs collaborative training on its original local model according to the collaborative training path and the local model sent by the data provider alliance node, obtains the collaboratively trained local model, builds its own local composite data set, and calculates the comprehensive evaluation score of the collaboratively trained local model based on the local composite data set; the comprehensive evaluation score is used to indicate whether the task initiator alliance node performs personalized training on the collaboratively trained local model, and the local composite data set is used to calculate the comprehensive evaluation score of the local model;

[0009] If the comprehensive evaluation score is less than the preset score threshold, the task initiator alliance node performs personalized training on the local model after collaborative training to obtain the final local model corresponding to the task initiator alliance node; otherwise, the task initiator alliance node uses the local model after collaborative training as the final local model.

[0010] Optionally, the link quality corresponding to each data alliance node is calculated according to the communication information, including:

[0011] By calculating the formula

[0012]

[0013]

[0014]

[0015]

[0016] ;

[0017] Get the first The link quality corresponding to the data alliance node ;in, Indicates the total length of the queue. The total length of the queue affects the efficiency and timeliness of the data alliance node in processing external requests. represents the arrival rate, which is the average rate of requests received by each data federation node, measured per unit time, represents the average queue length, Indicates The number of service windows of a data alliance node, Indicates the utilization rate of the system. Represents the system idle probability, which is used to describe the probability that all service windows or servers are idle at any given point in time. Indicates the number of customers or requests in the queue, Indicates The arrival rate of data alliance nodes, Indicates The service rate of each data alliance node.

[0018] Optionally, determining a collaborative training path according to link quality includes:

[0019] According to the link quality, the original link quality matrix is ​​constructed; the expression of the original link quality matrix is:

[0020] ;in, Indicates Link quality evaluation of each alliance node itself;

[0021] By calculating the formula

[0022] ;

[0023] Get the updated first Line Elements of a column ;in, represents the time step, represents an attenuation factor between 0 and 1, Indicates the number of data union nodes in the multi-data union node, Indicates that among the multiple data alliance nodes, except Data alliance nodes other than the data alliance node have The average value of the link quality evaluation of the data alliance nodes, Indicates that except for Except for the data alliance node, all other alliances have The average value of the link quality evaluation of the data alliance nodes;

[0024] By calculating the formula

[0025]

[0026]

[0027]

[0028]

[0029] Get the collaborative training path ;in, Represents candidate co-training paths The total network cost, represents the number of layers in the network, represents the weight factor, represents the network layer state matrix, Indicates the number of routing hops. Indicated in Layer, from Data Alliance Node to The link quality of each data alliance node, Indicates that from Data Alliance Node to The latency of each data alliance node, Indicates Data Alliance Node to The bandwidth of the data alliance nodes, Indicates Data Alliance Node to The square of the bandwidth of the data alliance nodes, Represents a composite indicator.

[0030] Optionally, the task initiator alliance node performs collaborative training on its original local model according to the collaborative training path and the local model sent by the data provider alliance node to obtain a collaboratively trained local model, including:

[0031] By calculating the formula

[0032] ;

[0033] Get the local model after collaborative training ;in, Indicated in Time steps, in the optimized training order The The model status of the alliance nodes, For the previous time step The model status, represents the learning rate, represents a soft target, compared to a hard target, Representation and optimized training sequence Middle A dataset of alliance nodes. Represents the temperature parameter, which controls the smoothness of the output probability distribution.

[0034] Optionally, build your own local composite dataset, including:

[0035] By calculating the formula

[0036]

[0037]

[0038]

[0039]

[0040] Get local composite dataset ;in, Represents a collection of time tags, represents a collection of spatial labels, Represents a set of parameters used for random sampling express, represents the dataset after time filtering, Indicates that at a given time Next, according to the space label After further filtering of the data set, represents the original data set, Representation dataset A specific data point in Indicates that it is applied to data points to extract its spatial label, Represents a specific space label. Indicates a specific time tag. represents random parameters, Indicates category.

[0041] Optionally, a comprehensive evaluation score of the co-trained local model is calculated based on the local composite dataset, including:

[0042] By calculating the formula

[0043] ;

[0044] Get the evaluation score of the local model after collaborative training ;in, represents the loss function, represents the global model at time t;

[0045] By calculating the formula

[0046]

[0047] Get the evaluation score of the original local model ;in, Indicates that at time step When, according to the optimized training sequence The The model status of each data union node;

[0048] By calculating the formula , and obtain the comprehensive evaluation score .

[0049] Optionally, perform personalized training on the local model after collaborative training, including:

[0050] Step I, by calculating the formula

[0051]

[0052]

[0053] Get the local model after personalized training ;in, Indicates that the allocation is in the order of optimization Middle Data sets of data alliance nodes, represents the adjustment coefficient, which is used to reflect the sensitivity of local data to model adjustment. It represents the learning rate, which is used to adjust the update amplitude. represents the weight parameter, Represents the gradient of the loss function with respect to the model parameters;

[0054] Step II: If the local model after personalized training meets the preset training termination condition, the local model after personalized training is used as the final local model; otherwise, As in step I , return to step I.

[0055] Optionally, the preset training termination condition is that the personalized effect evaluation value of the local model after personalized training is greater than or equal to a preset threshold.

[0056] Optionally, the calculation formula for the personalized effect evaluation value is as follows:

[0057]

[0058] in, Indicates the personalized effect evaluation value.

[0059] Optionally, the local model sent by the alliance node of the data provider has the same function as the original local model of the alliance node of the task initiator, but has different parameters.

[0060] The above solution of the present application has the following beneficial effects:

[0061] The collaborative training method based on multiple data alliances provided in the present application calculates the link quality of each data alliance node through the information computing party alliance node to determine the collaborative training path. The task initiator alliance node performs collaborative training on its own original local model according to the collaborative training path and the local model sent by the data provider alliance node. It does not rely on the global model aggregation of the central server, and realizes collaborative training of multiple data alliances in the absence of a central server, which is beneficial to improving the accuracy of the local model in the data alliance node. At the same time, the task initiator alliance node performs personalized training on the local model after collaborative training to prevent overfitting of the local model and effectively improve the accuracy of the local model.

[0062] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0064] Figure 1 This is a flow chart of a collaborative training method based on multiple data alliances provided in one embodiment of the present application. DETAILED DESCRIPTION

[0065] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0066] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0067] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0068] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0069] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0070] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0071] In view of the problem that traditional methods cannot realize collaborative training between multiple data alliance nodes without a centralized advanced server for data aggregation, resulting in low accuracy of local models in data alliance nodes, the present application provides a collaborative training method based on multiple data alliances, wherein the method calculates the link quality of each data alliance node through the information calculation party alliance node, determines the collaborative training path, and the task initiator alliance node performs collaborative training on its own original local model according to the collaborative training path and the local model sent by the data provider alliance node, without relying on the global model aggregation of the central server, thereby realizing collaborative training of multiple data alliance nodes without a central server, which is beneficial to improving the accuracy of local models in data alliance nodes; at the same time, the task initiator alliance node performs personalized training on the local model after collaborative training to prevent overfitting of the local model and effectively improve the accuracy of the local model.

[0072] The collaborative training method based on multiple data alliance provided in this application is described in detail below.

[0073] In an embodiment of the present application, a multi-data alliance (composed of multiple data alliance nodes) is divided into a task initiator alliance node, a data provider alliance node, and an information calculation party alliance node. Among them, the task initiator alliance node represents a data alliance node that needs to be collaboratively trained (specifically, it can be a server of the task initiator), the data provider alliance node provides the task initiator alliance node with the data required for collaborative training, the information calculation party alliance node is used to perform information calculation, the data provider alliance node includes at least one data alliance node, the distance between the information calculation party alliance node and each data alliance node in the multi-data alliance is less than a preset distance threshold, and the task initiator alliance node and the information calculation party alliance node are two different data alliance nodes in the multi-data alliance. Exemplarily, different data alliance nodes can correspond to servers in different large exchanges, and the server stores a risk transaction data identification model for risk transaction data identification. Due to the privacy of the data, there is often no central server for scheduling and aggregation between the data alliance nodes. However, the local data of each exchange itself is uneven. In order to ensure the accuracy of its own local model, it is necessary to use the data model of other exchanges for collaborative training.

[0074] The following is a description of the process of determining an information computing party alliance node from multiple data alliance nodes in an embodiment of the present application.

[0075] First, the distance between each data alliance node and other data alliance nodes except the task initiator alliance node among multiple data alliance nodes is calculated.

[0076] Specifically, the distance calculation formula is: .in, Indicates The physical distance between the data alliance nodes (except the data alliance nodes of the task initiator alliance) and the task initiator alliance nodes .

[0077] Then, the party alliance node is calculated based on the distance determination information.

[0078] Specifically, by calculating the formula ,Will The corresponding data alliance node is determined as the information computing party alliance node.

[0079] like Figure 1 As shown, the collaborative training method based on multiple data alliances includes the following steps:

[0080] Step 11: The task initiator alliance node sends a collaborative training request to the information computing party alliance node.

[0081] The task initiating data alliance initiates training requests to other alliances that provide data based on the local task. The local data of these alliances that provide data meet the specific requirements of the task initiating data alliance task, whether it is data quality, data quantity, category labels, etc., because usually the local data of the task initiating data alliance is insufficient, while the data of other alliances that provide data is sufficient.

[0082] Step 12: After receiving the collaborative training request, the information computing party alliance node collects the communication information of each data alliance node in the multi-data alliance, calculates the link quality corresponding to each data alliance node according to the communication information, determines the collaborative training path according to the link quality, and sends the collaborative training path to the task initiator alliance node.

[0083] The above communication information includes an arrival rate used to describe the average rate at which each data union node receives requests, a service rate used to describe the average rate at which each data union node processes requests, and the number of service windows used to process requests in each data union node. The collaborative training path indicates the order in which the data union nodes in the data provider's union node perform collaborative training.

[0084] Specifically, the link quality corresponding to each data alliance node is calculated based on the communication information, including:

[0085] By calculating the formula

[0086]

[0087]

[0088]

[0089]

[0090]

[0091] Get the first The link quality corresponding to the data alliance node ;in, Indicates the total length of the queue. The total length of the queue affects the efficiency and timeliness of the data alliance node in processing external requests. represents the arrival rate, which is the average rate of requests received by each data federation node, measured per unit time, represents the average queue length, Indicates The number of service windows of a data alliance node, Indicates the utilization rate of the system. Represents the system idle probability, which is used to describe the probability that all service windows or servers are idle at any given point in time. Indicates the number of customers or requests in the queue, Indicates The arrival rate of data alliance nodes, Indicates The service rate of each data alliance node.

[0092] The process of determining the collaborative training path according to the link quality in step 12 is described below.

[0093] Specifically, firstly, an original link quality matrix is ​​constructed according to the link quality.

[0094] In an embodiment of the present application, the expression of the original link quality matrix is:

[0095] .in, Indicates The link quality evaluation of each alliance node itself.

[0096] Then, by calculating the formula

[0097]

[0098] Get the updated first Line Elements of a column .

[0099] in, represents the time step, represents an attenuation factor between 0 and 1, Indicates the number of data union nodes in the multi-data union node, Indicates that in multiple data alliances, except Data alliance nodes other than the data alliance node have The average value of the link quality evaluation of the data alliance nodes, Indicates that except for Except for the data alliance node, all other alliances have The average value of the link quality evaluation of the data alliance nodes.

[0100] Finally, by calculating the formula

[0101]

[0102]

[0103]

[0104]

[0105] Get the collaborative training path .

[0106] in, Represents candidate co-training paths The total network cost, represents the number of layers in the network, represents the weight factor, represents the network layer state matrix, Indicates the number of routing hops. Indicated in Layer, from Data Alliance Node to The link quality of each data alliance node, Indicates that from Data Alliance Node to The latency of each data alliance node, Indicates Data Alliance Node to The bandwidth of the data alliance nodes, Indicates Data Alliance Node to The square of the bandwidth of the data alliance nodes, Represents a composite indicator.

[0107] In step 13, the task initiator alliance node performs collaborative training on its original local model according to the collaborative training path and the local model sent by the data provider alliance node to obtain the collaboratively trained local model, build its own local composite data set, and calculate the comprehensive evaluation score of the collaboratively trained local model based on the local composite data set.

[0108] The above comprehensive evaluation score is used to indicate whether the task initiator alliance node performs personalized training on the local model after collaborative training.

[0109] The local model sent by the above-mentioned data provider alliance node and the original local model of the task initiator alliance node itself have the same function (for example, the risk transaction data identification model of the exchange can be a decision tree model), but the parameters of the local model sent by the data provider alliance node and the original local model of the task initiator alliance node itself are different.

[0110] The following describes the process of collaboratively training the original local model according to the collaborative training path and the local model sent by the alliance node of the data provider in step 13 to obtain the collaboratively trained local model.

[0111] Specifically, by calculating the formula

[0112] ;

[0113] Get the local model after collaborative training ;in, Indicated in Time steps, in the optimized training order The The model status of the alliance nodes, For the previous time step The model status, represents the learning rate, represents a soft target, compared to a hard target, Representation and optimized training sequence Middle A dataset of alliance nodes. Represents the temperature parameter, which controls the smoothness of the output probability distribution.

[0114] It should be noted that the soft target represents the model's predicted probability distribution for each sample classification in the dataset. This probability distribution not only shows the most likely category predicted by the model, but also retains prediction information about other categories. This prediction information does not exist in ordinary hard targets (i.e. direct category labels). For example, in a model that classifies cats, dogs, and birds, the soft target for a particular image may be [0.8, 0.15, 0.05], while the hard target will only be [1, 0, 0] (if the correct category is a cat).

[0115] The process of constructing the local composite data set in step 13 is described below.

[0116] Specifically, by calculating the formula

[0117]

[0118]

[0119]

[0120]

[0121] Get local composite dataset ;in, Represents a collection of time tags, represents a collection of spatial labels, represents the set of parameters used for random sampling, express, represents the dataset after time filtering, Indicates that at a given time Next, according to the space label After further filtering of the data set, represents the original data set, Representation dataset A specific data point in Indicates that it is applied to data points to extract its spatial label, Represents a specific space tag. Indicates a specific time tag. represents random parameters, Indicates category.

[0122] In an embodiment of the present application, the local composite data set is used to calculate the comprehensive evaluation score of the local model.

[0123] Exemplarily, when the above local model and the original local model are both risk transaction data identification models of the exchange (such as a random forest model), the local composite data set is shown in the following table:

[0124] Table 1

[0125]

[0126] In the above table, transaction ID: unique identifier;

[0127] Transaction time: timestamp;

[0128] transaction amount: numeric value;

[0129] Transaction location: geographic location tag, such as New York, Tokyo, etc.;

[0130] Trader ID: unique identifier of the trader;

[0131] Stock code: the stock code;

[0132] Transaction type: buy, sell, etc.

[0133] The following is an explanation of the process of calculating the comprehensive evaluation score of the local model after collaborative training based on the local composite data set in step 13.

[0134] Specifically, by calculating the formula

[0135]

[0136] Get the evaluation score of the local model after collaborative training ;in, represents the loss function, represents the global model at time t. For example, the loss function The expression is as follows:

[0137] ;

[0138]

[0139] in, Is a student model The original output score of , after being adjusted by temperature T.

[0140] By calculating the formula

[0141]

[0142] Get the evaluation score of the original local model ;in, Indicates that at time step When, according to the optimized training sequence The The model status of each data union node;

[0143] By calculating the formula , and obtain the comprehensive evaluation score .

[0144] Step 14: If the comprehensive evaluation score is less than the preset score threshold, the task initiator alliance node performs personalized training on the collaboratively trained local model to obtain the final local model corresponding to the task initiator alliance node. Otherwise, the task initiator alliance node uses the collaboratively trained local model as the final local model.

[0145] In an embodiment of the present application, the preset training termination condition is that the personalized effect evaluation value of the local model after personalized training is greater than or equal to a preset threshold (for example, 0.2). The calculation formula of the personalized effect evaluation value is as follows:

[0146]

[0147] in, Indicates the personalized effect evaluation value.

[0148] The process of performing personalized training on the local model after collaborative training in step 14 is described below.

[0149] Step I, by calculating the formula

[0150]

[0151]

[0152] Get the local model after personalized training ;in, Indicates that the allocation is in the order of optimization Middle Data sets of data alliance nodes, represents the adjustment coefficient, which is used to reflect the sensitivity of local data to model adjustment. It represents the learning rate, which is used to adjust the update amplitude. represents the weight parameter, Represents the gradient of the loss function with respect to the model parameters.

[0153] Step II: If the local model after personalized training meets the preset training termination condition, the local model after personalized training is used as the final local model; otherwise, As in step I , return to step I.

[0154] Assuming we have activities in two different big data exchanges in Shanghai and New York, the global model provides a good baseline, but due to differences in regional markets (such as trading rules, market behaviors and preferences, etc.), local adjustments are needed.

[0155] Step 1: Apply multi-dimensional collaborative training method

[0156] Local composite dataset generation: Combine local data based on different labels in time and space to ensure that region-specific market dynamics are reflected.

[0157] Federated learning loop: Under the premise of ensuring data privacy, use federated learning methods for incremental learning and combine the advantages of other data alliances to optimize local models.

[0158] Step 2: Analyze the global model information gain and make personalized adjustments

[0159] Performance Evaluation: Compare the performance of the global model and the model fine-tuned with preliminary localization on the local dataset.

[0160] Information entropy gain analysis: Analyze the information gain of the global model in different local environments and identify the parts of the model that need further localization.

[0161] Step 3: Local Personalization Implementation

[0162] Personalized training process: Adjust model parameters to reflect the specific characteristics and needs of the local market. This may include adjusting the risk assessment model to adapt to local regulatory requirements and market volatility characteristics.

[0163] Establishment of feedback mechanism: Set up a system feedback mechanism to monitor model performance and make dynamic adjustments based on market changes

[0164] At the Shanghai Stock Exchange, a global model is used to adjust the stock index volatility prediction model for the Chinese market. Using a combination of local economic data and global economic indicators, a multi-dimensional collaborative training method is used to ensure that the model can capture China's unique market reaction patterns. After deployment, the model helps traders make more accurate trading decisions in an uncertain market environment, such as rapid response to changes in macroeconomic policies.

[0165] It can be seen from the above steps that the collaborative training method based on multiple data alliances provided by the present application calculates the link quality of each data alliance node through the information computing party alliance node, determines the collaborative training path, and the task initiator alliance node performs collaborative training on its own original local model according to the collaborative training path and the local model sent by the data provider alliance node. It does not rely on the global model aggregation of the central server, and realizes collaborative training of multiple data alliances in the absence of a central server, which is beneficial to improving the accuracy of the local model in the data alliance node; at the same time, the task initiator alliance node performs personalized training on the local model after collaborative training to prevent overfitting of the local model and effectively improve the accuracy of the local model.

[0166] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A collaborative training method based on multi-data alliance, characterized in that: The multi-data alliance includes a task initiator alliance node, a data provider alliance node and an information computing party alliance node, the data provider alliance node includes at least one data alliance node, the distance between the information computing party alliance node and each data alliance node in the multi-data alliance is less than a preset distance threshold, and the task initiator alliance node and the information computing party alliance node are two different data alliance nodes in the multi-data alliance; The collaborative training method comprises: The task initiator alliance node sends a collaborative training request to the information computing party alliance node; After receiving the collaborative training request, the information computing party alliance node collects communication information of each data alliance node in the multiple data alliances, calculates the link quality corresponding to each data alliance node according to the communication information, determines a collaborative training path according to the link quality, and sends the collaborative training path to the task initiator alliance node; the communication information includes an arrival rate for describing the average rate at which each data alliance node receives requests, a service rate for describing the average rate at which each data alliance node processes requests, and a number of service windows for describing the number of service windows for processing requests in each data alliance node; the collaborative training path indicates the order in which the data alliance nodes in the data provider alliance node perform collaborative training; The task initiator alliance node performs collaborative training on its own original local model according to the collaborative training path and the local model sent by the data provider alliance node to obtain a collaboratively trained local model, constructs its own local composite data set, and calculates a comprehensive evaluation score of the collaboratively trained local model according to the local composite data set; the comprehensive evaluation score is used to indicate whether the task initiator alliance node performs personalized training on the collaboratively trained local model, and the local composite data set is used to calculate the comprehensive evaluation score of the collaboratively trained local model; If the comprehensive evaluation score is less than the preset score threshold, the task initiator alliance node performs personalized training on the collaboratively trained local model to obtain the final local model corresponding to the task initiator alliance node; otherwise, the task initiator alliance node uses the collaboratively trained local model as the final local model.

2. The collaborative training method according to claim 1, characterized in that: The calculating the link quality corresponding to each data alliance node according to the communication information includes: By calculating the formula Get the first The link quality corresponding to the data alliance node ;in, Indicates the total length of the queue. The total length of the queue affects the efficiency and timeliness of the data alliance node in processing external requests. represents the arrival rate, which is the average rate of requests received by each data union node, measured per unit time, represents the average queue length, Indicates The number of service windows of a data alliance node, Indicates the utilization rate of the system. Represents the system idle probability, which is used to describe the probability that all service windows or servers are idle at any given point in time. Indicates the number of customers or requests in the queue, Indicates The arrival rate of data alliance nodes, Indicates The service rate of each data alliance node.

3. The collaborative training method according to claim 2, characterized in that: The determining of the collaborative training path according to the link quality includes: According to the link quality, an original link quality matrix is ​​constructed; the expression of the original link quality matrix is: ;in, Indicates Link quality evaluation of each alliance node itself; By calculating the formula Get the updated first Line Elements of a column ;in, represents the time step, represents an attenuation factor between 0 and 1, represents the number of data union nodes in the multiple data union nodes, Indicates that among the multiple data alliance nodes, except the Data alliance nodes other than the data alliance node have The average value of the link quality evaluation of the data alliance nodes, Indicates that except for Except for the data alliance node, all other alliances have The average value of the link quality evaluation of the data alliance nodes; By calculating the formula Get the collaborative training path ;in, Represents candidate co-training paths The total network cost, represents the number of layers in the network, represents the weight factor, represents the network layer state matrix, Indicates the number of routing hops. Indicated in Layer, from Data Alliance Node to The link quality of each data alliance node, Indicates that from Data Alliance Node to The latency of each data alliance node, Indicates Data Alliance Node to The bandwidth of the data alliance nodes, Indicates Data Alliance Node to The square of the bandwidth of the data alliance nodes, Represents a composite indicator.

4. The collaborative training method according to claim 3, characterized in that: The task initiator alliance node performs collaborative training on its own original local model according to the collaborative training path and the local model sent by the data provider alliance node to obtain a collaboratively trained local model, including: By calculating the formula ; Get the local model after the collaborative training ;in, Indicated in Time steps, in the optimized training order The The model status of the alliance nodes, For the previous time step The model status, represents the learning rate, represents a soft target, Representation and optimized training sequence Middle A dataset of alliance nodes. Represents the temperature parameter, which controls the smoothness of the output probability distribution.

5. The collaborative training method according to claim 4, characterized in that: The construction of its own local composite data set includes: By calculating the formula Get the local composite dataset ;in, Represents a collection of time tags, represents a collection of spatial labels, represents the set of parameters used for random sampling, express, represents the dataset after time filtering, Indicates that at a given time Next, according to the space label After further filtering of the data set, represents the original data set, Representation dataset A specific data point in Indicates that it is applied to data points to extract its spatial label, Represents a specific space label. Indicates a specific time tag. represents random parameters, Indicates category.

6. The collaborative training method according to claim 5, characterized in that: The calculating the comprehensive evaluation score of the collaboratively trained local model according to the local composite data set includes: By calculating the formula Get the evaluation score of the local model after the collaborative training ;in, represents the loss function, Indicates at time The global model of By calculating the formula Get the evaluation score of the original local model ;in, Indicates at time When, according to the optimized training sequence The The model status of each data union node; By calculating the formula , and obtain the comprehensive evaluation score .

7. The collaborative training method according to claim 6, characterized in that: The performing personalized training on the collaboratively trained local model includes: Step I, by calculating the formula Get the local model after personalized training ;in, Indicates that the allocation is in the order of optimization Middle Data sets of data alliance nodes, Represents the adjustment coefficient, which is used to reflect the sensitivity of local data to model adjustment. It represents the learning rate, which is used to adjust the update amplitude. represents the weight parameter, Represents the gradient of the loss function with respect to the model parameters; Step II: If the local model after the personalized training meets the preset training termination condition, the local model after the personalized training is used as the final local model; otherwise, As in step I , return to step I.

8. The collaborative training method according to claim 7, characterized in that: The preset training termination condition is that the personalized effect evaluation value of the local model after the personalized training is greater than or equal to a preset threshold.

9. The collaborative training method according to claim 8, characterized in that: The calculation formula of the personalized effect evaluation value is as follows: in, Represents the personalized effect evaluation value.

10. The collaborative training method according to claim 1, characterized in that: The local model sent by the data provider alliance node has the same function as the original local model of the task initiator alliance node itself, but has different parameters.

Citation Information

Patent Citations

  • Mobile overlay virtual enterprise network and virtual internet for enterprises

    US20190230032A1

  • Data-sharing systems and methods, which use multi-angle incentive allocation

    US20230015813A1