A protocol selection system, method, apparatus, device, and storage medium

Abstract

Embodiments of the present application provide a protocol selection system, method, device, equipment and storage medium, and relate to the technical field of computer, which comprises an initiating device and a plurality of participating devices. The initiating device sends candidate aggregation protocols to each participating device. The participating device trains a to-be-trained model by using locally stored sample data to obtain training parameters, processes the training parameters according to a parameter processing mode specified by each candidate aggregation protocol, and sends the processing result to the initiating device. The initiating device obtains the processing result corresponding to each candidate aggregation protocol sent by each participating device, aggregates the obtained processing result by using each candidate aggregation protocol, adjusts the to-be-trained model by using the aggregation result, obtains the evaluation parameters of the adjusted to-be-trained model, and selects a target aggregation protocol from the plurality of candidate aggregation protocols, wherein the corresponding evaluation parameters of the target aggregation protocol satisfy a preset evaluation condition. The application of the system can improve the reliability of the model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a protocol selection system, method, apparatus, device, and storage medium. Background Technology

[0002] In traditional machine learning, training is typically centralized, meaning that sample data is concentrated on the initiating device where the model to be trained resides for training. If the original sample data is not on the initiating device, other participating devices need to send sample data to the initiating device. The initiating device may obtain the privacy information of the participating devices based on the sent sample data. Therefore, to protect privacy, each participating device uses its local sample data to train the same model as the one on the initiating device and sends the training results to the initiating device. The initiating device selects a aggregation protocol to aggregate the received training results, obtaining the aggregated result, and uses the aggregated result to adjust the model to be trained on the initiating device.

[0003] In this case, if the aggregation method of the selected aggregation protocol yields different aggregation results for different training results, the training effect of the model will be different. If the aggregation result does not perform well in training the model, it may cause the model to fail to converge, resulting in poor model reliability. Summary of the Invention

[0004] The purpose of this application is to provide a protocol selection system, method, apparatus, device, and storage medium to improve the reliability of the model. The specific technical solution is as follows:

[0005] This application provides a protocol selection system, which includes an initiating device and multiple participating devices, wherein:

[0006] The initiating device is configured to: obtain multiple candidate convergence protocols and send the candidate convergence protocols to each participating device;

[0007] The participating device is configured to: train the model to be trained using locally stored sample data to obtain training parameters; process the training parameters according to the parameter processing method specified by the candidate convergence protocol for each candidate convergence protocol to obtain the processing result; and send the processing result to the initiating device.

[0008] The initiating device is configured to: for each candidate convergence protocol, obtain the processing results corresponding to the candidate convergence protocol sent by each participating device, converge the obtained processing results using the candidate convergence protocol to obtain a convergence result, adjust the model to be trained using the convergence result to obtain the evaluation parameters of the adjusted model to be trained, and select a target convergence protocol from multiple candidate convergence protocols whose evaluation parameters meet preset evaluation conditions, wherein the target convergence protocol is used to continue training the model to be trained.

[0009] This application embodiment also provides a protocol selection method applied to an initiating device, the method comprising:

[0010] Obtain multiple candidate convergence protocols and send the candidate convergence protocols to each participating device;

[0011] The system receives processing results sent by each participating device. Different candidate convergence protocols correspond to different processing results. For each candidate convergence protocol, the processing result is: after the participating device trains the model to be trained using the sample data stored locally to obtain training parameters, it processes the training parameters according to the parameter processing method specified by the candidate convergence protocol.

[0012] For each candidate convergence protocol, the processing results corresponding to the candidate convergence protocol sent by each participating device are obtained. The obtained processing results are converged using the candidate convergence protocol to obtain the convergence result. The model to be trained is adjusted using the convergence result to obtain the evaluation parameters of the adjusted model to be trained. A target convergence protocol whose evaluation parameters meet the preset evaluation conditions is selected from multiple candidate convergence protocols. The target convergence protocol is used to continue training the model to be trained.

[0013] This application embodiment also provides a protocol selection method applied to participating devices, the method comprising:

[0014] Receive candidate aggregation protocols sent by the initiating device, wherein the candidate aggregation protocols are: multiple candidate aggregation protocols obtained by the initiating device;

[0015] The model to be trained is trained using locally stored sample data to obtain training parameters. For each candidate aggregation protocol, the training parameters are processed according to the parameter processing method specified by that candidate aggregation protocol to obtain the processing result, and the processing result is sent to the initiating device.

[0016] This allows the initiating device to obtain the processing results corresponding to each candidate convergence protocol sent by each participating device for each candidate convergence protocol, to aggregate the obtained processing results using the candidate convergence protocol to obtain a convergence result, to adjust the model to be trained using the convergence result to obtain the evaluation parameters of the adjusted model to be trained, and to select a target convergence protocol from multiple candidate convergence protocols whose evaluation parameters meet preset evaluation conditions, wherein the target convergence protocol is used to continue training the model to be trained.

[0017] This application embodiment also provides a protocol selection device, applied to an initiating device, the device comprising:

[0018] The protocol sending module is used to obtain multiple candidate convergence protocols and send the candidate convergence protocols to each participating device.

[0019] The result receiving module is used to receive the processing results sent by each participating device. Different candidate convergence protocols correspond to different processing results. For each candidate convergence protocol, the processing result corresponding to the candidate convergence protocol is: after the participating device uses the sample data stored locally to train the model to be trained to obtain training parameters, it processes the training parameters according to the parameter processing method specified by the candidate convergence protocol.

[0020] The protocol selection module is used to obtain the processing results corresponding to each candidate convergence protocol sent by each participating device for each candidate convergence protocol, to converge the obtained processing results using the candidate convergence protocol to obtain the convergence result, to adjust the model to be trained using the convergence result to obtain the evaluation parameters of the adjusted model to be trained, and to select the target convergence protocol whose evaluation parameters meet the preset evaluation conditions from multiple candidate convergence protocols, wherein the target convergence protocol is used to continue training the model to be trained.

[0021] Optionally, the protocol selection module includes:

[0022] The result acquisition unit is used to determine the contribution of each participating device for each candidate convergence protocol, and to perform weighted calculation on the processing results corresponding to each participating device using the determined contribution as the weight, so as to obtain the weighted processing result.

[0023] The parameter acquisition unit is used to aggregate the weighted processing results using the candidate protocol to obtain the aggregated result, and to adjust the model to be trained using the aggregated result to obtain the evaluation parameters of the adjusted model to be trained.

[0024] The protocol selection unit is used to select a target convergence protocol from multiple candidate convergence protocols whose evaluation parameters meet preset evaluation conditions, wherein the target convergence protocol is used to continue training the model to be trained.

[0025] Optionally, the result obtaining unit is specifically used for:

[0026] For each candidate aggregation protocol and for each participating device, a first ratio is determined between the amount of sample data locally on that participating device and the total amount of data; a second ratio is determined between the current training round of the model to be trained by that participating device and the total number of rounds; the similarity between the processing result sent by that participating device and the reference aggregation result is determined, wherein the total amount of data is the sum of the amount of sample data locally on all participating devices, the total number of rounds is the number of rounds in which the initiating device adjusts the parameters of the model to be trained, and the reference aggregation result is the aggregation result used by the initiating device when adjusting the parameters of the model to be trained in the previous round; based on the determined first ratio, second ratio, and similarity, the contribution of each participating device is determined; using the determined contribution as the weight, the processing results corresponding to each participating device are weighted and calculated to obtain the weighted processing result.

[0027] Optionally, the parameter acquisition unit is specifically used to aggregate the encrypted processing results sent by each participating device using the candidate aggregation protocol to obtain the aggregation result, and to adjust the model to be trained using the aggregation result to obtain the evaluation parameters of the adjusted model to be trained.

[0028] Optionally, the evaluation parameters include at least one of the following:

[0029] The accuracy of the model to be trained after parameter adjustment, the time required for the model to be trained to reach the preset convergence condition, the amount of data exchanged between the initiating device and each participating device, and the number of training rounds required for the model to be trained to reach the preset convergence condition.

[0030] Optionally, different evaluation parameters have different priorities;

[0031] The protocol selection unit is specifically used for:

[0032] The process involves: determining the highest priority first evaluation parameter; selecting a reference convergence protocol from multiple candidate convergence protocols whose first evaluation parameter satisfies preset evaluation conditions; if the number of selected reference convergence protocols is greater than 1, determining a second evaluation parameter with the next highest priority after the first evaluation parameter, updating the first evaluation parameter to the second evaluation parameter, updating the candidate convergence protocol to the reference convergence protocol, and returning to the step of selecting a reference convergence protocol from multiple candidate convergence protocols whose first evaluation parameter satisfies preset evaluation conditions, until the number of selected reference convergence protocols is 1, and then determining the selected reference convergence protocol as the target convergence protocol.

[0033] Optionally, the device further includes:

[0034] The first digest generation module is used to receive registration information sent by each participating device; for each participating device, it calculates the registration information sent by the participating device according to a preset digest generation method, generates the first digest information corresponding to the participating device, and sends the digest generation method to the participating device.

[0035] The digest receiving module is used to receive second digest information sent by each participating device, wherein the second digest information is: information generated by each participating device based on the local registration information calculated by the digest generation method received by the digest generation method;

[0036] The device verification module is used to determine whether the verification of each participating device is successful if the first summary information and the second summary information of the participating device match.

[0037] This application embodiment also provides a protocol selection device, applied to participating equipment, the device comprising:

[0038] The protocol receiving module is used to receive candidate convergence protocols sent by the initiating device, wherein the candidate convergence protocols are: multiple candidate convergence protocols obtained by the initiating device;

[0039] The model training module is used to train the model to be trained using locally stored sample data to obtain training parameters. For each candidate convergence protocol, the training parameters are processed according to the parameter processing method specified by the candidate convergence protocol to obtain a processing result. The processing result is then sent to the initiating device so that the initiating device can obtain the processing results corresponding to the candidate convergence protocol sent by each participating device for each candidate convergence protocol. The obtained processing results are then converged using the candidate convergence protocol to obtain a convergence result. The convergence result is used to adjust the model to be trained to obtain the evaluation parameters of the adjusted model to be trained. A target convergence protocol that meets the preset evaluation conditions is selected from multiple candidate convergence protocols. The target convergence protocol is used to continue training the model to be trained.

[0040] Optionally, the model training module is specifically used to train the model to be trained using locally stored sample data to obtain training parameters; for each candidate convergence protocol, the training parameters are processed according to the parameter processing method specified by the candidate convergence protocol to obtain a processing result; after obtaining the processing result, the obtained processing result is encrypted using a homomorphic encryption algorithm, and the encrypted processing result is sent to the initiating device.

[0041] Optionally, the device further includes:

[0042] The information sending module is used to send registration information to the initiating device;

[0043] The second digest generation module is used to receive the digest generation method, calculate the local registration information based on the digest generation method, generate second digest information, and send the second digest information to the initiating device, so that the initiating device determines that the participating device has passed verification if the first digest information and the second digest information of the participating device match.

[0044] The digest generation method is as follows: the initiating device receives registration information sent by each participating device; for each participating device, the device calculates the registration information sent by the participating device according to a preset digest generation method to generate a first digest information corresponding to the participating device, and sends the digest generation method to the participating device.

[0045] This application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0046] Memory, used to store computer programs;

[0047] When a processor executes a program stored in memory, it implements the protocol selection method steps described in any of the preceding statements.

[0048] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the protocol selection method steps described above.

[0049] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the protocol selection method steps described in any of the above embodiments.

[0050] The beneficial effects of the embodiments of this application are as follows:

[0051] In the solution provided in this application embodiment, the initiating device obtains processing results from each participating device and aggregates these results to obtain an aggregated result. Since the aggregated result includes the processing results of training with sample data from multiple participating devices, and the models to be trained on each participating device are consistent with those on the initiating device, training the model to be trained using the aggregated result is equivalent to referencing the sample data from all participating devices, resulting in sufficient sample data. Furthermore, when selecting the aggregation protocol, a superior aggregation protocol is chosen based on evaluation parameters, ensuring the training effect of the aggregated result on the model training. In this case, both the amount of training sample data on the initiating device and the training effect are guaranteed, improving the reliability of the resulting trained model. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0053] Figure 1 This is a schematic diagram of a protocol selection system provided in an embodiment of this application.

[0054] Figure 2 This is a schematic diagram of the structure of a device communication system provided in an embodiment of this application.

[0055] Figure 3 This is a schematic diagram of a registration information verification process provided in an embodiment of this application.

[0056] Figure 4 This is a flowchart illustrating the first protocol selection method provided in an embodiment of this application.

[0057] Figure 5This is a flowchart illustrating a second protocol selection method provided in an embodiment of this application.

[0058] Figure 6 This is a schematic diagram of the structure of a first protocol selection device provided in an embodiment of this application.

[0059] Figure 7 This is a schematic diagram of the structure of a second protocol selection device provided in an embodiment of this application.

[0060] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of the present invention.

[0062] The application scenarios of the embodiments of this application will be described below.

[0063] In one application scenario, this application can train the model to be trained on the initiating device using sample data distributed across participating devices, thus realizing the federated learning process. In this case, the initiating device completes the training without directly obtaining the original sample data from each participating device, following the method steps of the subsequent embodiments, thereby protecting the privacy of each participating device.

[0064] On the other hand, the sample data used in the embodiments of this application can be a small amount of sample data from the sample dataset used for pre-training. Thus, in the following embodiments, each participating device uses a small amount of sample data for training, reducing the workload of model training. The training performed at this time is pre-training, used to determine the optimal convergence protocol.

[0065] After selecting a convergence protocol according to the scheme provided in the embodiments of this application, the selected convergence protocol can be used to train a model on a large amount of sample data.

[0066] In one embodiment of this application, see Figure 1 A schematic diagram of a protocol selection system is provided, which includes an initiating device and participating devices. As shown in the figure, there can be multiple participating devices, and each participating device, namely participating device 1, participating device 2, ..., participating device N in the figure, can perform the protocol selection steps in the manner shown in the following embodiments.

[0067] The initiating device and participating device can be various types of electronic devices, such as desktop computers and servers.

[0068] The initiating device is used to obtain multiple candidate convergence protocols and send the candidate convergence protocols to each participating device.

[0069] The participating device uses locally stored sample data to train the model to be trained, obtains training parameters, processes the training parameters according to the parameter processing method specified by each candidate convergence protocol, obtains the processing result, and sends the processing result to the initiating device.

[0070] The initiating device is used to: for each candidate aggregation protocol, obtain the processing results corresponding to the candidate aggregation protocol sent by each participating device, aggregate the obtained processing results using the candidate aggregation protocol to obtain the aggregation result, adjust the model to be trained using the aggregation result to obtain the evaluation parameters of the adjusted model to be trained, and select the target aggregation protocol from multiple candidate aggregation protocols whose evaluation parameters meet the preset evaluation conditions, wherein the target aggregation protocol is used to continue training the model to be trained.

[0071] The aforementioned initiating and participating devices are equipped with the same training model, which can be a face recognition model, object segmentation model, license plate recognition model, target event detection model, etc.

[0072] A convergence protocol is a protocol used to converge parameters obtained from training various participating devices.

[0073] When training a model, the method of using the processing results varies depending on the type of processing results obtained. For example, if the obtained result is a gradient, which is an adjustment value used to adjust the parameters of the training model, then the gradient is aggregated according to the aggregation protocol, and the aggregated gradient is used to adjust the parameters of the model to be trained. The adjusted parameters represent the weight coefficients of various data processing in the model to be trained. If the obtained result is a parameter value, which is the coefficient value of the weight coefficient, then the parameter value is aggregated according to the aggregation protocol, and the aggregated parameter value is used to replace the original parameter value in the model to be trained.

[0074] The candidate aggregation protocol can contain a variety of different aggregation protocols for protocol selection. For example, the aggregation protocols it contains can be FedAVG (Federated Averaging) protocol, FedProx (Federated proximal) protocol, etc.

[0075] Participating devices can execute the steps described above for obtaining and sending processing results in multiple rounds. In this case, the initiating device can receive processing results from multiple rounds and aggregate them across multiple rounds. Specific aggregation methods are detailed in subsequent embodiments and will not be elaborated here.

[0076] The following describes the specific implementation method of the participating device training the model to be trained locally in the above embodiments.

[0077] Training a model can involve inputting sample data into the model and having it output the training result. The specific training result varies depending on the type of model. For example, if the model is a classification model, the training result could be the classification probability; if it's an image recognition model, the result could be the identity of a person in an image. Training parameters can be obtained based on the difference between the training result and the expected output of the model. Specifically, these parameters can be the gradient generated by the objective function that calculates the difference, or the model parameters adjusted based on the gradient. The objective function can be set according to the following formula:

[0078]

[0079] Where θ represents the weights of the model parameters that need to be adjusted in the model. The above training parameters are represented. This represents sample data.

[0080] Candidate convergence protocols can specify parameter processing methods to manipulate the obtained training parameters and implement parameter adjustment strategies. For example, the quantity of sample data affects the model training effect; if the quantity of sample data is too small, the accuracy of the model to be trained will be low, and the effectiveness of the training results will be insufficient. Accordingly, the parameter adjustment strategy can be to adjust the impact of the training results of the participating devices according to the quantity of sample data. The corresponding parameter processing method is shown in the following formula:

[0081]

[0082] In the formula, For the processing results, Let n be the training parameters on the i-th participating device. In the formula, assuming there are k participating devices, the value of i can be any integer from 1, 2, ..., k, and n is the training parameters on the i-th participating device. k N represents the total amount of sample data contained in all participating devices during the current training round. i This represents the total number of sample data on the i-th participating device.

[0083] When participating devices send processing results to the initiating device, to ensure that the processing results are not obtained by other devices not in the protocol selection system due to plaintext transmission, the processing results can be encrypted before being sent. The specific implementation method is as follows:

[0084] In one implementation, after receiving the processing result, the participating device can encrypt the result using a homomorphic encryption algorithm and send the encrypted result to the initiating device. During transmission, the processing result is homomorphic ciphertext.

[0085] In another implementation, a symmetric encryption algorithm can be used to encrypt the processing result, but this application does not limit this approach.

[0086] The following describes the specific implementation method of the initiating device aggregating the processing results in the aforementioned embodiments.

[0087] Each candidate aggregation protocol can specify an aggregation method, and then aggregates all the aggregation results received by the initiating device using the specified aggregation method to obtain the aggregation result of each candidate aggregation protocol. Different candidate aggregation protocols specify different aggregation methods.

[0088] For example, in the FedAVG aggregation protocol, the specified aggregation method can be to take the average of the processing results. In other aggregation protocols, the specified aggregation method can be to calculate a weighted average, median, maximum, etc. When calculating the weighted average, weights can be set based on the contribution of each participating device, as described in subsequent embodiments, which will not be detailed here.

[0089] If each processing result is ciphertext, then during aggregation, the obtained ciphertext needs to be decrypted accordingly to obtain the correct aggregation result. For example, if each processing result is homomorphic ciphertext, aggregation can be performed first based on the homomorphic ciphertext processing results. The candidate aggregation protocol can then be used to aggregate the encrypted processing results sent by each participating device to obtain the aggregation result. In this case, the resulting aggregation result is homomorphic ciphertext. This homomorphic ciphertext aggregation result is then decrypted to obtain the plaintext aggregation result.

[0090] Therefore, using homomorphic encryption algorithms to transmit the processing results ensures the security of the results during transmission.

[0091] In the above process, adjusting the model to be trained and selecting a target convergence protocol from multiple candidate convergence protocols can be viewed as a pre-training process for the model. Conversely, continuing to train the model using the selected target convergence protocol can involve receiving processing results from new participating devices, aggregating these results using the target convergence protocol, obtaining a new convergence result, adjusting the model using this new result, obtaining the evaluation parameters of the adjusted model, and finally, obtaining the trained model. The specific implementation is similar to the method of adjusting the model to be trained in the above embodiments, the only difference being the convergence protocol used, which will not be detailed here.

[0092] As can be seen from the above, in the solution provided by this application embodiment, the initiating device obtains the processing results from each participating device and aggregates these results to obtain an aggregated result. Since the aggregated result contains the processing results of training with sample data from multiple participating devices, and the models to be trained on each participating device are consistent with those on the initiating device, training the model to be trained using the aggregated result is equivalent to referencing the sample data from all participating devices, resulting in sufficient sample data. Furthermore, when selecting the aggregation protocol, a superior aggregation protocol is chosen based on the evaluation parameters, ensuring the training effect of the aggregated result on the model training. In this case, both the amount of training sample data on the initiating device and the training effect are guaranteed, thus improving the reliability of the resulting trained model.

[0093] The following describes the specific implementation method of the initiating device adjusting the model to be trained in the aforementioned embodiments.

[0094] The model to be trained can be independently adjusted based on the aggregation result obtained from each candidate aggregation protocol, resulting in multiple adjusted models. The training performance of the models trained using different candidate aggregation protocols varies, and the training performance can be determined based on the following evaluation parameters:

[0095] The accuracy of the model to be trained after parameter adjustment, the time required for the model to reach the preset convergence condition, the amount of data exchanged between the initiating device and each participating device, and the number of training rounds required for the model to reach the preset convergence condition.

[0096] The accuracy of the model to be trained can be determined by the difference between the output of the model to be trained and the expected result; the preset convergence condition can be that the difference between the output of the model to be trained and the expected result is less than a preset threshold, etc.

[0097] As can be seen from the above, the evaluation parameters take into account information from multiple aspects during the training process, and the information referenced for evaluating the training effect of the convergence protocol is relatively sufficient. The evaluation parameters provide a more comprehensive and accurate assessment of the training effect.

[0098] In one embodiment of this application, priorities can be assigned to various evaluation parameters, and the final selected convergence protocol can be determined based on whether the evaluation conditions are met for different priority evaluation parameters. This convergence protocol serves as the target convergence protocol for subsequent training of the model to be trained. Specific implementation details are provided in subsequent embodiments and will not be elaborated here.

[0099] In the above embodiments, the initiating device and participating devices communicate to enable the participants to obtain the aggregation result according to the above steps. For specific communication methods, see [link to documentation]. Figure 2 The example shown.

[0100] like Figure 2 The node management module provides participant information to partyA (the initiator), i.e., the initiating device. In the initiating device's algorithm implementation, the `fit` module adjusts the device's training model, and the `set_param` module sets scene parameters, including port parameters for participating devices. This allows the initiating device to send information to the ports of participating devices according to the port parameters—essentially sending remote computing tasks—so that participating devices, i.e., partyB (the participant), can train their training models using locally stored sample data. The partyA (initiator)'s aggregation protocol is used to obtain the aggregation results, and the `evaluate` module sets the objective function.

[0101] In the algorithm implementation of the participating device, after receiving the remote computing task, the fit module is used to train the model to be trained on the participating device, and the set_param module is used to set the scene parameters, including the port parameters of the initiating device, so that the participating device can send messages to the port of the initiating device and feed back the remote computing results, that is, the processing results obtained by the parameter processing method specified by the convergence protocol of partyB-participant in the aforementioned embodiment; the evaluation of the participating device is used to set the objective function for local model training.

[0102] The routing communication module is used to discover routes and transmit various types of messages sent by participating and initiating devices. For example, during the process of partyB (participant) reporting remote calculation results, the routing communication module receives the reported calculation results and reports the remote calculation results to partyA (initiator).

[0103] The following describes the specific implementation method of setting weights based on the contribution of each participating device in the aforementioned embodiments and obtaining the aggregation result.

[0104] In one embodiment of this application, the contribution of each participating device can be determined, and the processing results corresponding to each participating device can be weighted using the determined contribution as the weight to obtain the weighted processing result; the weighted processing result can be aggregated using the candidate protocol to obtain the aggregated result.

[0105] The convergence result is obtained using the following formula:

[0106]

[0107] Where γ i This represents the contribution of the i-th participating device; The processing result of the i-th participating device. To aggregate the results.

[0108] As can be seen from the above, by weighting the contribution, the processing results of participating devices with higher contribution have a greater impact on the convergence results, which is more in line with the actual training situation of each participating device in the actual training process, and can improve the effectiveness of the obtained convergence results.

[0109] Specifically, the degree of contribution can be determined as follows:

[0110] For each participating device, a first ratio is determined between the amount of sample data locally on that participating device and the total amount of data; a second ratio is determined between the current training round of the model to be trained on that participating device and the total number of rounds; and the similarity between the processing result sent by that participating device and the reference aggregation result is determined. The total amount of data is the sum of the amount of sample data locally on all participating devices, the total number of rounds is the number of rounds in which the initiating device adjusted the parameters of the model to be trained, and the reference aggregation result is the aggregation result used by the initiating device when adjusting the parameters of the model to be trained in the previous round.

[0111] The contribution of each participating device is determined based on the established first ratio, second ratio, and similarity.

[0112] The above calculation process can be expressed by the following formula:

[0113]

[0114] Where, γ i The value represents the contribution, where i indicates the contribution of the i-th participating device, meaning it can represent the contribution of any participating device. (dataScale) i The totalScale represents the amount of sample data from the i-th participating device, and totalScale represents the total amount of data, i.e., the total amount of sample data from all participating devices; aggreTimes i This indicates the round in which the i-th participating device adjusts the model to be trained, aggreTimest This indicates the total number of rounds that have been performed so far. The processing result of the i-th participating device. This refers to the aggregation result from the previous round, i.e., the reference aggregation result. In the formula, the similarity is calculated as follows: and The cosine similarity.

[0115] As can be seen from the above, multiple parameters of the participating equipment were considered when determining the contribution, and the information referenced when describing the contribution was more comprehensive, which can improve the accuracy of the obtained contribution.

[0116] In one embodiment of this application, different evaluation parameters have different priorities.

[0117] In this case, in the aforementioned embodiment, the target convergence protocol is obtained according to the following steps D1-D3:

[0118] Step D1: Determine the highest priority first evaluation parameter;

[0119] Step D2: Select a reference convergence protocol from multiple candidate convergence protocols whose first evaluation parameters satisfy the preset evaluation conditions;

[0120] The preset evaluation conditions can be that the parameter value of the first evaluation parameter of the selected candidate convergence protocol is greater than the parameter value of the first evaluation parameter of another candidate convergence protocol, and the difference in parameter values ​​between the two candidate convergence protocols being compared is greater than a preset threshold.

[0121] Step D3: If the number of selected reference convergence protocols is greater than 1, determine the second evaluation parameter with the next priority after the first evaluation parameter, update the first evaluation parameter to the second evaluation parameter, update the candidate convergence protocol to the reference convergence protocol, and return to step D2 to select the reference convergence protocol whose first evaluation parameter meets the preset evaluation conditions from multiple candidate convergence protocols, until the number of selected reference convergence protocols is 1, and determine the selected reference convergence protocol as the target convergence protocol.

[0122] According to the preset evaluation conditions of D2, the selection can result in two situations: one is that there is a candidate convergence protocol that meets the evaluation conditions for all other convergence protocols, then the number of selected reference convergence protocols is 1, and this candidate convergence protocol is the target convergence protocol.

[0123] In another scenario, if there are multiple candidate convergence protocols that all meet the evaluation criteria, but do not meet the evaluation criteria for each other (i.e., the parameter values ​​under the first evaluation parameter are relatively close), then the first evaluation parameter is updated using the second evaluation parameter with the next higher priority, and the comparison is performed again.

[0124] If, after comparing all priority evaluation parameters, there is still more than one convergence protocol, then one of these convergence protocols can be randomly selected as the target convergence protocol.

[0125] As can be seen from the above, screening target convergence protocols according to priority makes the evaluation parameters with higher priority have a greater effect on the selection of target convergence protocols. In fact, the evaluation parameters with higher priority have higher value in evaluating the training effect during actual training. Accordingly, the convergence results obtained by the target convergence protocols selected accordingly have better training effects.

[0126] In one embodiment of this application, the participating device is further configured to: send registration information to the initiating device;

[0127] The registration information may include the amount of sample data from the participating devices during training, so that the initiating device can count the amount of data sent by each participating device to obtain the total amount of data in the aforementioned embodiment.

[0128] In addition, the registration information may also include the IP address of the participating device; the data structure of the sample data, including the data type of the sample data and the field information of the data table storing the sample data; and the computing power of the participating device, including GPU (graphics processing unit) computing power, CPU (central processing unit) computing power, and memory size.

[0129] The initiating device is also used to: receive registration information sent by each participating device; for each participating device, calculate the registration information sent by the participating device according to a preset digest generation method, generate the first digest information corresponding to the participating device, and send the first digest information to the participating device to send the digest generation method to the participating device;

[0130] The above digest generation method can be a hash algorithm, such as MD5, SHA-1, etc.; the registration information of the participating devices is used as the input parameter of the hash algorithm, and the output parameter is the first digest information.

[0131] The participating device is also used to: receive the digest generation method, calculate the local registration information based on the digest generation method, generate second digest information, and send the digest information to the initiating device;

[0132] The method for generating the second summary information is similar to that for the first summary information, with the only difference being the substitution of names such as second summary information and first summary information, which will not be detailed here.

[0133] The initiating device is also used to: for each participating device, determine that the participating device has passed verification if the first digest information and the second digest information of the participating device match.

[0134] If the first and second summary information match, it means the registration information has not been modified, and the participating device still retains the attributes set in its registration information. The initiating device can then assign corresponding training tasks to the participating device according to these attributes, enabling it to begin training the model to be trained in any round. In this case, the participating device is deemed to have passed verification.

[0135] As can be seen from the above, the initiating device determines the participating devices by verifying the registration information, which ensures that all participating devices are subject to security verification when the protocol selection system is established, making the established system more secure.

[0136] Specifically, the flowchart for the above registration information verification is as follows: Figure 3 As shown.

[0137] like Figure 3 The process involves several steps: A participant submits a registration application, indicating that the participating devices are sending registration information; the initiator accepts the registration, indicating that it has received the registration information from each participating device; the initiator generates a summary, namely the first summary, and sends a summary generation method; the participating devices receive the summary generation method and generate their own summaries, namely the second summary; and the participating devices send their summaries to the initiator, allowing the initiator to compare the summary information, i.e., match the first and second summary information. If a participating device matches, the registration information is considered tamper-proof, and in this case, the registration information format can be verified; if the registration information has been tampered with, the process ends, meaning the participating device's registration fails.

[0138] During the format review, if the format is incorrect, the process ends; if the format is correct, a participant service connectivity test is performed, in which the initiating device communicates with the participating device to determine if the participating device's port can connect normally. If the connectivity test fails, the process ends; if the connectivity test succeeds, the initiator writes the registration information, and the registration is successful.

[0139] In this case, when performing any of the protocol selection steps in the foregoing embodiments, a successfully registered participating device can be used to improve the security of protocol selection.

[0140] Corresponding to the above-described protocol selection system, in one embodiment of this application, see [link to relevant documentation]. Figure 4 The flowchart of the first protocol selection method is provided and applied to the initiating device, including the following steps S401-S403.

[0141] Step S401: Obtain multiple candidate convergence protocols and send the candidate convergence protocols to each participating device;

[0142] Step S402: Receive the processing results sent by each participating device;

[0143] Different candidate convergence protocols correspond to different processing results. For each candidate convergence protocol, the processing result is: after the participating device uses the sample data stored locally to train the model to be trained and obtains the training parameters, it processes the training parameters according to the parameter processing method specified by the candidate convergence protocol.

[0144] Step S403: For each candidate aggregation protocol, obtain the processing results corresponding to the candidate aggregation protocol sent by each participating device, aggregate the obtained processing results using the candidate aggregation protocol to obtain the aggregation result, adjust the model to be trained using the aggregation result, obtain the evaluation parameters of the adjusted model to be trained, and select the target aggregation protocol whose evaluation parameters meet the preset evaluation conditions from multiple candidate aggregation protocols.

[0145] The target convergence protocol is used to continue training the model to be trained.

[0146] As can be seen from the above, in the solution provided by this application embodiment, the initiating device obtains the processing results from each participating device and aggregates these results to obtain an aggregated result. Since the aggregated result contains the processing results of training with sample data from multiple participating devices, and the models to be trained on each participating device are consistent with those on the initiating device, training the model to be trained using the aggregated result is equivalent to referencing the sample data from all participating devices, resulting in sufficient sample data. Furthermore, when selecting the aggregation protocol, a superior aggregation protocol is chosen based on the evaluation parameters, ensuring the training effect of the aggregated result on the model training. In this case, both the amount of training sample data on the initiating device and the training effect are guaranteed, thus improving the reliability of the resulting trained model.

[0147] In one embodiment of this application, the step of using the candidate aggregation protocol to aggregate the obtained processing results to obtain an aggregation result includes:

[0148] Determine the contribution of each participating device, and use the determined contribution as the weight to calculate the weighted processing result for each participating device.

[0149] The weighted processing results are aggregated using this candidate protocol to obtain the aggregated result.

[0150] As can be seen from the above, by weighting the contribution, the processing results of participating devices with higher contribution have a greater impact on the convergence results, which is more in line with the actual training situation of each participating device in the actual training process, and can improve the effectiveness of the obtained convergence results.

[0151] In one embodiment of this application, determining the contribution of each participating device includes:

[0152] For each participating device, a first ratio is determined between the amount of sample data locally on that participating device and the total amount of data; a second ratio is determined between the current training round of the model to be trained by that participating device and the total number of rounds; the similarity between the processing result sent by that participating device and the reference aggregation result is determined, wherein the total amount of data is the sum of the amount of sample data locally on all participating devices, the total number of rounds is the number of rounds in which the initiating device adjusts the parameters of the model to be trained, and the reference aggregation result is the aggregation result used by the initiating device when adjusting the parameters of the model to be trained in the previous round;

[0153] The contribution of each participating device is determined based on the established first ratio, second ratio, and similarity.

[0154] As can be seen from the above, multiple parameters of the participating equipment were considered when determining the contribution, and the information referenced when describing the contribution was more comprehensive, which can improve the accuracy of the obtained contribution.

[0155] In one embodiment of this application, obtaining the processing results corresponding to the candidate aggregation protocol sent by each participating device, and aggregating the obtained processing results using the candidate aggregation protocol to obtain the aggregation result includes:

[0156] The participating device receives the processing result sent by the participating device, wherein the processing result is the result obtained by the participating device after obtaining the processing result and encrypting the obtained processing result using a homomorphic encryption algorithm;

[0157] The encrypted processing results sent by each participating device are aggregated using this candidate aggregation protocol to obtain the aggregation result.

[0158] Therefore, using homomorphic encryption algorithms to transmit the processing results ensures the security of the results during transmission.

[0159] In one embodiment of this application, the evaluation parameters include at least one of the following:

[0160] The accuracy of the model to be trained after parameter adjustment, the time required for the model to be trained to reach the preset convergence condition, the amount of data exchanged between the initiating device and each participating device, and the number of training rounds required for the model to be trained to reach the preset convergence condition.

[0161] As can be seen from the above, the evaluation parameters take into account information from multiple aspects during the training process, and the information referenced for evaluating the training effect of the convergence protocol is relatively sufficient. The evaluation parameters provide a more comprehensive and accurate assessment of the training effect.

[0162] In one embodiment of this application, different evaluation parameters have different priorities;

[0163] The step of selecting a target convergence protocol from multiple candidate convergence protocols whose evaluation parameters satisfy preset evaluation conditions includes:

[0164] Determine the highest priority first evaluation parameter;

[0165] Select a reference convergence protocol from multiple candidate convergence protocols whose first evaluation parameters satisfy the preset evaluation conditions;

[0166] If the number of selected reference convergence protocols is greater than 1, determine a second evaluation parameter that has the next priority after the first evaluation parameter, update the first evaluation parameter to the second evaluation parameter, update the candidate convergence protocol to the reference convergence protocol, and return to the step of selecting a reference convergence protocol whose corresponding first evaluation parameter meets the preset evaluation conditions from multiple candidate convergence protocols, until the number of selected reference convergence protocols is 1, and determine the selected reference convergence protocol as the target convergence protocol.

[0167] As can be seen from the above, screening target convergence protocols according to priority makes the evaluation parameters with higher priority have a greater effect on the selection of target convergence protocols. In fact, the evaluation parameters with higher priority have higher value in evaluating the training effect during actual training. Accordingly, the convergence results obtained by the target convergence protocols selected accordingly have better training effects.

[0168] In one embodiment of this application, before obtaining multiple candidate convergence protocols and sending the candidate convergence protocols to each participating device, the method further includes:

[0169] Receive registration information sent by each participating device; for each participating device, calculate the registration information sent by the participating device according to the preset digest generation method, generate the first digest information corresponding to the participating device, and send the digest generation method to the participating device;

[0170] Receive second digest information sent by each participating device, wherein the second digest information is: information generated by each participating device based on the digest generation method and its local registration information calculated by the digest generation method;

[0171] For each participating device, if the first and second summary information of the participating device match, the participating device is determined to have passed the verification.

[0172] As can be seen from the above, the initiating device determines the participating devices by verifying the registration information, which ensures that all participating devices are subject to security verification when the protocol selection system is established, making the established system more secure.

[0173] Corresponding to the above-described protocol selection system, in one embodiment of this application, see [link to relevant documentation]. Figure 5 The flowchart of the second protocol selection method is provided and applied to participating devices, including the following steps S501-S502.

[0174] Step S501: Receive the candidate aggregation protocol sent by the initiating device.

[0175] The candidate convergence protocol is: a plurality of candidate convergence protocols obtained by the initiating device;

[0176] Step S502: Train the model to be trained using the sample data stored locally to obtain training parameters. For each candidate convergence protocol, process the training parameters according to the parameter processing method specified by the candidate convergence protocol to obtain the processing result, and send the processing result to the initiating device.

[0177] Step S502 above enables the initiating device to obtain the processing results corresponding to each candidate convergence protocol sent by each participating device for each candidate convergence protocol, to converge the obtained processing results using the candidate convergence protocol to obtain the convergence result, to adjust the model to be trained using the convergence result to obtain the evaluation parameters of the adjusted model to be trained, and to select the target convergence protocol whose evaluation parameters meet the preset evaluation conditions from multiple candidate convergence protocols.

[0178] The target convergence protocol is used to continue training the model to be trained.

[0179] As can be seen from the above, in the solution provided by this application embodiment, the initiating device obtains the processing results from each participating device and aggregates these results to obtain an aggregated result. Since the aggregated result contains the processing results of training with sample data from multiple participating devices, and the models to be trained on each participating device are consistent with those on the initiating device, training the model to be trained using the aggregated result is equivalent to referencing the sample data from all participating devices, resulting in sufficient sample data. Furthermore, when selecting the aggregation protocol, a superior aggregation protocol is chosen based on the evaluation parameters, ensuring the training effect of the aggregated result on the model training. In this case, both the amount of training sample data on the initiating device and the training effect are guaranteed, thus improving the reliability of the resulting trained model.

[0180] In one embodiment of this application, sending the processing result to the initiating device includes:

[0181] After obtaining the processing result, the obtained processing result is encrypted using a homomorphic encryption algorithm, and the encrypted processing result is sent to the initiating device.

[0182] Therefore, using homomorphic encryption algorithms to transmit the processing results ensures the security of the results during transmission.

[0183] In one embodiment of this application, before receiving the candidate convergence protocol sent by the initiating device, the method further includes:

[0184] Send registration information to the initiating device;

[0185] The system receives the digest generation method, calculates the local registration information based on the digest generation method, generates second digest information, and sends the second digest information to the initiating device, so that the initiating device determines that the participating device has passed verification if the first digest information and the second digest information of the participating device match.

[0186] The digest generation method is as follows: the initiating device receives registration information sent by each participating device; for each participating device, the device calculates the registration information sent by the participating device according to a preset digest generation method to generate a first digest information corresponding to the participating device, and sends the digest generation method to the participating device.

[0187] As can be seen from the above, the initiating device determines the participating devices by verifying the registration information, which ensures that all participating devices are subject to security verification when the protocol selection system is established, making the established system more secure.

[0188] Corresponding to the above method embodiments, this application also provides a protocol selection device.

[0189] In one embodiment of this application, see Figure 6 A schematic diagram of a first protocol selection device is provided, which is applied to an initiating device. The device includes:

[0190] Protocol sending module 601 is used to obtain multiple candidate convergence protocols and send the candidate convergence protocols to each participating device;

[0191] The result receiving module 602 is used to receive the processing results sent by each participating device. Different candidate convergence protocols correspond to different processing results. For each candidate convergence protocol, the processing result corresponding to the candidate convergence protocol is: the result obtained by the participating device after training the training model using the sample data stored locally to obtain the training parameters, and then processing the training parameters according to the parameter processing method specified by the candidate convergence protocol.

[0192] The protocol selection module 603 is used to obtain the processing results corresponding to the candidate convergence protocol sent by each participating device for each candidate convergence protocol, to converge the obtained processing results using the candidate convergence protocol to obtain a convergence result, to adjust the model to be trained using the convergence result to obtain the evaluation parameters of the adjusted model to be trained, and to select the target convergence protocol whose evaluation parameters meet the preset evaluation conditions from multiple candidate convergence protocols, wherein the target convergence protocol is used to continue training the model to be trained.

[0193] As can be seen from the above, in the solution provided by this application embodiment, the initiating device obtains the processing results from each participating device and aggregates these results to obtain an aggregated result. Since the aggregated result contains the processing results of training with sample data from multiple participating devices, and the models to be trained on each participating device are consistent with those on the initiating device, training the model to be trained using the aggregated result is equivalent to referencing the sample data from all participating devices, resulting in sufficient sample data. Furthermore, when selecting the aggregation protocol, a superior aggregation protocol is chosen based on the evaluation parameters, ensuring the training effect of the aggregated result on the model training. In this case, both the amount of training sample data on the initiating device and the training effect are guaranteed, thus improving the reliability of the resulting trained model.

[0194] In one embodiment of this application, the protocol selection module 603 includes:

[0195] The result acquisition unit is used to determine the contribution of each participating device for each candidate convergence protocol, and to perform weighted calculation on the processing results corresponding to each participating device using the determined contribution as the weight, so as to obtain the weighted processing result.

[0196] The parameter acquisition unit is used to aggregate the weighted processing results using the candidate protocol to obtain the aggregated result, and to adjust the model to be trained using the aggregated result to obtain the evaluation parameters of the adjusted model to be trained.

[0197] The protocol selection unit is used to select a target convergence protocol from multiple candidate convergence protocols whose evaluation parameters meet preset evaluation conditions, wherein the target convergence protocol is used to continue training the model to be trained.

[0198] As can be seen from the above, by weighting the contribution, the processing results of participating devices with higher contribution have a greater impact on the convergence results, which is more in line with the actual training situation of each participating device in the actual training process, and can improve the effectiveness of the obtained convergence results.

[0199] In one embodiment of this application, the result obtaining unit is specifically used for:

[0200] For each candidate aggregation protocol and for each participating device, a first ratio is determined between the amount of sample data locally on that participating device and the total amount of data; a second ratio is determined between the current training round of the model to be trained by that participating device and the total number of rounds; the similarity between the processing result sent by that participating device and the reference aggregation result is determined, wherein the total amount of data is the sum of the amount of sample data locally on all participating devices, the total number of rounds is the number of rounds in which the initiating device adjusts the parameters of the model to be trained, and the reference aggregation result is the aggregation result used by the initiating device when adjusting the parameters of the model to be trained in the previous round; based on the determined first ratio, second ratio, and similarity, the contribution of each participating device is determined; using the determined contribution as the weight, the processing results corresponding to each participating device are weighted and calculated to obtain the weighted processing result.

[0201] As can be seen from the above, multiple parameters of the participating equipment were considered when determining the contribution, and the information referenced when describing the contribution was more comprehensive, which can improve the accuracy of the obtained contribution.

[0202] In one embodiment of this application, the parameter acquisition unit is specifically used to aggregate the encrypted processing results sent by each participating device using the candidate aggregation protocol to obtain an aggregation result, and to adjust the model to be trained using the aggregation result to obtain the evaluation parameters of the adjusted model to be trained.

[0203] Therefore, using homomorphic encryption algorithms to transmit the processing results ensures the security of the results during transmission.

[0204] In one embodiment of this application, the evaluation parameters include at least one of the following:

[0205] The accuracy of the model to be trained after parameter adjustment, the time required for the model to be trained to reach the preset convergence condition, the amount of data exchanged between the initiating device and each participating device, and the number of training rounds required for the model to be trained to reach the preset convergence condition.

[0206] As can be seen from the above, the evaluation parameters take into account information from multiple aspects during the training process, and the information referenced for evaluating the training effect of the convergence protocol is relatively sufficient. The evaluation parameters provide a more comprehensive and accurate assessment of the training effect.

[0207] In one embodiment of this application, different evaluation parameters have different priorities;

[0208] The protocol selection unit is specifically used for:

[0209] The process involves: determining the highest priority first evaluation parameter; selecting a reference convergence protocol from multiple candidate convergence protocols whose first evaluation parameter satisfies preset evaluation conditions; if the number of selected reference convergence protocols is greater than 1, determining a second evaluation parameter with the next highest priority after the first evaluation parameter, updating the first evaluation parameter to the second evaluation parameter, updating the candidate convergence protocol to the reference convergence protocol, and returning to the step of selecting a reference convergence protocol from multiple candidate convergence protocols whose first evaluation parameter satisfies preset evaluation conditions, until the number of selected reference convergence protocols is 1, and then determining the selected reference convergence protocol as the target convergence protocol.

[0210] As can be seen from the above, screening target convergence protocols according to priority makes the evaluation parameters with higher priority have a greater effect on the selection of target convergence protocols. In fact, the evaluation parameters with higher priority have higher value in evaluating the training effect during actual training. Accordingly, the convergence results obtained by the target convergence protocols selected accordingly have better training effects.

[0211] In one embodiment of this application, the apparatus further includes:

[0212] The first digest generation module is used to receive registration information sent by each participating device; for each participating device, it calculates the registration information sent by the participating device according to a preset digest generation method, generates the first digest information corresponding to the participating device, and sends the digest generation method to the participating device.

[0213] The digest receiving module is used to receive second digest information sent by each participating device, wherein the second digest information is: information generated by each participating device based on the local registration information calculated by the digest generation method received by the digest generation method;

[0214] The device verification module is used to determine whether the verification of each participating device is successful if the first summary information and the second summary information of the participating device match.

[0215] As can be seen from the above, the initiating device determines the participating devices by verifying the registration information, which ensures that all participating devices are subject to security verification when the protocol selection system is established, making the established system more secure.

[0216] In one embodiment of this application, see Figure 7 A schematic diagram of a second protocol selection device is provided, which is applied to participating equipment. The device includes:

[0217] The protocol receiving module 701 is used to receive candidate aggregation protocols sent by the initiating device, wherein the candidate aggregation protocols are: multiple candidate aggregation protocols obtained by the initiating device;

[0218] The model training module 702 is used to train the model to be trained using locally stored sample data to obtain training parameters. For each candidate convergence protocol, the training parameters are processed according to the parameter processing method specified by the candidate convergence protocol to obtain a processing result. The processing result is then sent to the initiating device so that the initiating device can obtain the processing results corresponding to the candidate convergence protocol sent by each participating device for each candidate convergence protocol. The obtained processing results are then converged using the candidate convergence protocol to obtain a convergence result. The convergence result is used to adjust the model to be trained to obtain the evaluation parameters of the adjusted model to be trained. A target convergence protocol that satisfies the preset evaluation conditions is selected from multiple candidate convergence protocols. The target convergence protocol is used to continue training the model to be trained.

[0219] As can be seen from the above, in the solution provided by this application embodiment, the initiating device obtains the processing results from each participating device and aggregates these results to obtain an aggregated result. Since the aggregated result contains the processing results of training with sample data from multiple participating devices, and the models to be trained on each participating device are consistent with those on the initiating device, training the model to be trained using the aggregated result is equivalent to referencing the sample data from all participating devices, resulting in sufficient sample data. Furthermore, when selecting the aggregation protocol, a superior aggregation protocol is chosen based on the evaluation parameters, ensuring the training effect of the aggregated result on the model training. In this case, both the amount of training sample data on the initiating device and the training effect are guaranteed, thus improving the reliability of the resulting trained model.

[0220] In one embodiment of this application, the model training module 702 is specifically used to train the model to be trained using locally stored sample data to obtain training parameters; for each candidate convergence protocol, the training parameters are processed according to the parameter processing method specified by the candidate convergence protocol to obtain a processing result; after obtaining the processing result, the obtained processing result is encrypted using a homomorphic encryption algorithm, and the encrypted processing result is sent to the initiating device.

[0221] Therefore, using homomorphic encryption algorithms to transmit the processing results ensures the security of the results during transmission.

[0222] In one embodiment of this application, the apparatus further includes:

[0223] The information sending module is used to send registration information to the initiating device;

[0224] The second digest generation module is used to receive the digest generation method, calculate the local registration information based on the digest generation method, generate second digest information, and send the second digest information to the initiating device, so that the initiating device determines that the participating device has passed verification if the first digest information and the second digest information of the participating device match.

[0225] The digest generation method is as follows: the initiating device receives registration information sent by each participating device; for each participating device, the device calculates the registration information sent by the participating device according to a preset digest generation method to generate a first digest information corresponding to the participating device, and sends the digest generation method to the participating device.

[0226] As can be seen from the above, the initiating device determines the participating devices by verifying the registration information, which ensures that all participating devices are subject to security verification when the protocol selection system is established, making the established system more secure.

[0227] This application also provides an electronic device, such as... Figure 8 As shown, it includes a processor 801, a communication interface 802, a memory 803, and a communication bus 804. The processor 801, communication interface 802, and memory 803 communicate with each other via the communication bus 804.

[0228] Memory 803 is used to store computer programs;

[0229] When the processor 801 executes the program stored in the memory 803, it implements the protocol selection method steps described in any of the foregoing embodiments:

[0230] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0231] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0232] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0233] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0234] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described protocol selection methods.

[0235] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the protocol selection methods described above.

[0236] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0237] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0238] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of methods, apparatus, electronic devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0239] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A protocol selection system, characterized by, The system comprises an initiating device and a plurality of participating devices, wherein: The initiating device is configured to: obtain a plurality of candidate aggregation protocols, and send the candidate aggregation protocols to the participating devices; The participating devices are configured to: train a to-be-trained model by using locally stored sample data to obtain training parameters, process the training parameters according to a parameter processing manner specified by each candidate aggregation protocol to obtain processing results, and send the processing results corresponding to each candidate aggregation protocol to the initiating device; The initiating device is configured to: obtain, for each candidate aggregation protocol, the processing results corresponding to the candidate aggregation protocol sent by the participating devices, aggregate the obtained processing results by using the candidate aggregation protocol to obtain aggregation results, adjust the to-be-trained model by using the aggregation results to obtain evaluation parameters of the to-be-trained model after adjustment, and select, from the plurality of candidate aggregation protocols, a target aggregation protocol corresponding to which the evaluation parameters satisfy a preset evaluation condition, wherein the target aggregation protocol is used for continuing to train the to-be-trained model.

2. The system of claim 1, wherein, The initiating device is specifically configured to: Determine the contribution degrees of the participating devices, and perform weighted calculation on the processing results corresponding to the participating devices by using the determined contribution degrees as weights to obtain weighted processing results; Aggregate the weighted processing results by using the candidate protocol to obtain aggregation results.

3. The system of claim 2, wherein, The initiating device is specifically configured to: For each participating device, determine a first proportion of a data amount of sample data local to the participating device to a total data amount, and determine a second proportion of a current round in which the participating device currently trains the to-be-trained model to a total round; Determine a similarity between the processing result sent by the participating device and a reference aggregation result, wherein the total data amount is a sum of data amounts of sample data local to all participating devices, the total round is a round in which the initiating device adjusts parameters of the to-be-trained model, and the reference aggregation result is an aggregation result used by the initiating device when adjusting parameters of the to-be-trained model in a previous round; Determine the contribution degree of each participating device according to the determined first proportion, second proportion and similarity.

4. The system of claim 1, wherein, The participating devices are specifically configured to: After obtaining the processing result, encrypt the obtained processing result by using a homomorphic encryption algorithm, and send the encrypted processing result to the initiating device; The initiating device is specifically configured to: Aggregate the encrypted processing results sent by the participating devices by using the candidate aggregation protocol to obtain aggregation results.

5. The system of claim 1, wherein, The evaluation parameters comprise at least one of: an accuracy of the to-be-trained model after parameter adjustment, a time required for the to-be-trained model to reach a preset convergence condition, a data amount of communication between the initiating device and the participating devices, and a training round required for the to-be-trained model to reach the preset convergence condition. Different evaluation parameters have different priorities; 6. The system of claim 5, wherein, The initiating device is specifically configured to: Determine a first evaluation parameter with the highest priority; Select, from the plurality of candidate aggregation protocols, a reference aggregation protocol corresponding to which the first evaluation parameter satisfies a preset evaluation condition. ​ In a case that the number of the selected reference convergence protocols is greater than 1, a second evaluation parameter in a next priority level of the first evaluation parameter is determined, the first evaluation parameter is updated to the second evaluation parameter, the candidate convergence protocol is updated to the reference convergence protocol, and the step of selecting the reference convergence protocol corresponding to the first evaluation parameter satisfying the preset evaluation condition from the plurality of candidate convergence protocols is returned until the number of the selected reference convergence protocols is 1, and the selected reference convergence protocol is determined as the target convergence protocol.

7. The system of any one of claims 1-6, wherein, The participating device is further configured to send the registration information to the initiating device. The initiating device is further configured to receive the registration information sent by each participating device, calculate the registration information sent by each participating device according to a preset digest generation method to generate first digest information corresponding to each participating device, and send the digest generation method to each participating device. The participating device is further configured to receive the digest generation method, calculate the local registration information based on the digest generation method to generate second digest information, and send the second digest information to the initiating device. The initiating device is further configured to determine that each participating device passes the verification in a case that the first digest information and the second digest information of the participating device match.

8. A protocol selection method characterized by, The method applied to the initiating device comprises: obtaining a plurality of candidate convergence protocols, and sending the candidate convergence protocols to each participating device; receiving processing results sent by each participating device, wherein different candidate convergence protocols correspond to different processing results, and for each candidate convergence protocol, the processing result corresponding to the candidate convergence protocol is a result obtained by processing the training parameters according to a parameter processing mode specified by the candidate convergence protocol after the participating devices train the to-be-trained model by using the locally stored sample data to obtain the training parameters; for each candidate convergence protocol, obtaining the processing result corresponding to the candidate convergence protocol sent by each participating device, converging the obtained processing result by using the candidate convergence protocol to obtain a convergence result, adjusting the to-be-trained model by using the convergence result, obtaining an evaluation parameter of the adjusted to-be-trained model, and selecting a target convergence protocol corresponding to the evaluation parameter satisfying a preset evaluation condition from the plurality of candidate convergence protocols, wherein the target convergence protocol is used for continuing to train the to-be-trained model.

9. A protocol selection method characterized by, The method applied to the participating device comprises: receiving the candidate convergence protocol sent by the initiating device, wherein the candidate convergence protocol is the plurality of candidate convergence protocols obtained by the initiating device; training the to-be-trained model by using the locally stored sample data to obtain training parameters, processing the training parameters according to the parameter processing mode specified by each candidate convergence protocol to obtain a processing result, and sending the processing result to the initiating device, The initiating device obtains, for each candidate aggregation protocol, processing results corresponding to the candidate aggregation protocol and sent by each participating device, aggregates the obtained processing results by using the candidate aggregation protocol, obtains an aggregation result, adjusts the to-be-trained model by using the aggregation result, and obtains evaluation parameters of the to-be-trained model after adjustment, and selects a target aggregation protocol corresponding to evaluation parameters satisfying a preset evaluation condition from the plurality of candidate aggregation protocols, wherein the target aggregation protocol is used for continuing training of the to-be-trained model.

10. A protocol selection apparatus characterized by comprising: The application is applied to an initiating device, and the device comprises: A protocol sending module is configured to obtain a plurality of candidate aggregation protocols and send the candidate aggregation protocols to each participating device. A result receiving module is configured to receive processing results sent by each participating device, wherein different candidate aggregation protocols correspond to different processing results, and for each candidate aggregation protocol, the processing result corresponding to the candidate aggregation protocol is a result obtained by processing training parameters after the participating device trains a to-be-trained model by using locally stored sample data according to a parameter processing mode specified by the candidate aggregation protocol. A protocol selection module is configured to, for each candidate aggregation protocol, obtain processing results corresponding to the candidate aggregation protocol and sent by each participating device, aggregate the obtained processing results by using the candidate aggregation protocol, obtain an aggregation result, adjust the to-be-trained model by using the aggregation result, obtain evaluation parameters of the to-be-trained model after adjustment, and select a target aggregation protocol corresponding to evaluation parameters satisfying a preset evaluation condition from the plurality of candidate aggregation protocols, wherein the target aggregation protocol is used for continuing training of the to-be-trained model.

11. A protocol selection apparatus characterized by comprising: The application is applied to a participating device, and the device comprises: A protocol receiving module is configured to receive a candidate aggregation protocol sent by an initiating device, wherein the candidate aggregation protocol is a plurality of candidate aggregation protocols obtained by the initiating device. A model training module is configured to train a to-be-trained model by using locally stored sample data, obtain training parameters, process the training parameters according to a parameter processing mode specified by each candidate aggregation protocol to obtain processing results, and send the processing results to the initiating device, so that the initiating device obtains, for each candidate aggregation protocol, processing results corresponding to the candidate aggregation protocol and sent by each participating device, aggregates the obtained processing results by using the candidate aggregation protocol, obtains an aggregation result, adjusts the to-be-trained model by using the aggregation result, obtains evaluation parameters of the to-be-trained model after adjustment, and selects a target aggregation protocol corresponding to evaluation parameters satisfying a preset evaluation condition from the plurality of candidate aggregation protocols, wherein the target aggregation protocol is used for continuing training of the to-be-trained model.

12. An electronic device, comprising: The application comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus. The memory is configured to store a computer program. The processor is configured to execute the program stored on the memory to implement the method steps in any one of claims 8 or 9.

13. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method steps in any one of claims 8 or 9.

Images