Model contribution degree evaluation method and device, computer device, medium and product

Abstract

The application relates to a model contribution degree evaluation method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: obtaining a first model accuracy of a federated learning model, obtaining a second model accuracy of the federated learning model, and calculating a model contribution degree of a first participant to the federated learning model according to a standard model accuracy of the federated learning model, the first model accuracy and the second model accuracy, and a data quality index of original sample data provided by the first participant and a data quality index of original sample data provided by a second participant. The data quality index of the original sample data can effectively adjust the model contribution degrees of the participants obtained according to the federated learning model, so that the model contribution degrees can better reflect the overall situation in a real application scenario, the calculation result tends to be closer to real data, and the purpose of reasonably distributing the benefits of the federated learning model among the participants is achieved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, computer equipment, medium, and product for evaluating model contribution. Background Technology

[0002] Currently, with increasing demands and awareness of data privacy and security, enterprises are increasingly employing vertical federated learning techniques for secure joint modeling. This not only ensures data security but also enables models to acquire more features and achieve greater accuracy. Simultaneously, machine learning models are trained using data from each participant, and the trained models are shared among all participants. At this point, each participant needs to participate in the distribution of model benefits based on their respective contributions during the model training process. Therefore, accurately evaluating each participant's model contribution ensures a more equitable distribution of benefits, thereby significantly promoting collaboration between enterprises.

[0003] Traditional techniques often calculate each participant's contribution based on the cumulative number of times the features provided by each participant are used during the actual joint modeling process, and then distribute model benefits according to the contribution. However, the cumulative number of times features are used does not accurately reflect the contribution of features to model training.

[0004] Therefore, the accuracy of the model contribution of each participant calculated using traditional techniques during the model training process is relatively low. Summary of the Invention

[0005] Therefore, it is necessary to provide a model contribution evaluation method, apparatus, computer equipment, medium, and product to address the aforementioned technical problems.

[0006] Firstly, this application provides a method for evaluating model contribution, wherein the model includes a federated learning model, the federated learning model includes a first participant and a second participant, the method is applied to the first participant in the federated learning model, and the method includes:

[0007] Obtain the first model accuracy of the federated learning model; the first model accuracy is the model accuracy corresponding to the first participant.

[0008] Obtain the second model accuracy of the federated learning model; the second model accuracy is the model accuracy corresponding to the second participant.

[0009] The contribution of the first participant to the federated learning model is calculated based on the standard model accuracy, the first model accuracy, and the second model accuracy, combined with the data quality index of the original sample data provided by the first participant and the data quality index of the original sample data provided by the second participant.

[0010] In one embodiment, obtaining the first model accuracy of the federated learning model includes:

[0011] Obtain the first original sample data and the first intermediate sample data of the first participant; the first intermediate sample data includes sample data obtained by perturbating the first original sample data.

[0012] Receive the second intermediate sample data from the second participant; the second intermediate sample data includes sample data obtained by perturbating the second original sample data from the second participant.

[0013] Each first original feature from the first original sample data and each second intermediate feature from the second intermediate sample data are input into the federated learning model, and the first model accuracy of the federated learning model corresponding to each first original feature is calculated.

[0014] In one embodiment, obtaining the second model accuracy of the federated learning model includes:

[0015] The second participant receives the second model accuracy of the federated learning model sent by the second participant to the first participant. The second model accuracy is the second model accuracy of the federated learning model corresponding to each of the second original features in the second original sample data and each of the first intermediate features in the first intermediate sample data, which is calculated by the second participant inputting each of the second original features in the second original sample data and each of the first intermediate features in the first intermediate sample data into the federated learning model.

[0016] In one embodiment, the original sample data provided by the first participant includes first original sample data, and the original sample data provided by the second participant includes second original sample data. The method further includes:

[0017] Obtain the data quality index of each of the first original features in the first original sample data;

[0018] Obtain the data quality index of each of the second original features in the second original sample data.

[0019] In one embodiment, the data quality index of the first original feature includes a feature validity index and / or a feature diversity index; obtaining the data quality index of each of the first original features in the first original sample data includes:

[0020] Obtain the feature effectiveness index of each of the first original features in the first original sample data; and / or

[0021] Obtain the feature diversity index of each of the first original features in the first original sample data;

[0022] For each of the first original features in the first original sample data, a data quality index for the first original feature is generated based on the feature validity index and / or the feature diversity index of the first original feature.

[0023] In one embodiment, calculating the model contribution of the first participant to the federated learning model based on the standard model accuracy, the first model accuracy, and the second model accuracy, combined with the data quality index of the original sample data provided by the first participant and the data quality index of the original sample data provided by the second participant, includes:

[0024] For each first original feature in the first original sample data, the model contribution of each first original feature is calculated based on the standard model accuracy of the federated learning model, the first model accuracy corresponding to the first original feature, the data quality index of the first original feature, the second model accuracy corresponding to each second original feature in the second original sample data, and the data quality index of each second original feature.

[0025] The model contribution of each first original feature in the first original sample data is summed to obtain the model contribution of the first original sample data to the federated learning model.

[0026] The contribution of the first original sample data to the federated learning model is taken as the contribution of the first participant to the federated learning model.

[0027] In one embodiment, the step of calculating the model contribution of each first original feature for each first original feature in the first original sample data, based on the standard model accuracy of the federated learning model, the first model accuracy corresponding to the first original feature, and the data quality index of the first original feature, and the second model accuracy corresponding to each second original feature in the second original sample data and the data quality index of each second original feature, includes:

[0028] For each first original feature in the first original sample data, calculate the product of the first model accuracy corresponding to the first original feature and the data quality index of the first original feature;

[0029] Calculate the first model accuracy and the first weighted sum of the data quality indices corresponding to all first original features in the first original sample data; calculate the second model accuracy and the second weighted sum of the data quality indices corresponding to all second original features in the second original sample data.

[0030] The first weighted sum and the second weighted sum are summed to generate a target weighted sum. The quotient of the product corresponding to each of the first original features and the target weighted sum is calculated to generate the model contribution of each of the first original features.

[0031] Secondly, this application also provides a model contribution evaluation device, wherein the model includes a federated learning model, the federated learning model includes a first participant and a second participant, the device is applied to the first participant in the federated learning model, and the device includes:

[0032] The first model accuracy acquisition module is used to acquire the first model accuracy of the federated learning model; the first model accuracy is the model accuracy corresponding to the first participant.

[0033] The second model accuracy acquisition module is used to acquire the second model accuracy of the federated learning model; the second model accuracy is the model accuracy corresponding to the second participant.

[0034] The model contribution calculation module is used to calculate the model contribution of the first participant to the federated learning model based on the standard model accuracy, the first model accuracy, and the second model accuracy of the federated learning model, combined with the data quality index of the original sample data provided by the first participant and the data quality index of the original sample data provided by the second participant.

[0035] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the model contribution evaluation method as described above.

[0036] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the model contribution evaluation method as described above.

[0037] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the model contribution evaluation method as described above.

[0038] The aforementioned model contribution evaluation method, apparatus, computer equipment, storage medium, and computer program product are used to obtain the first model accuracy of the federated learning model, wherein the first model accuracy is the model accuracy corresponding to the first participant. A second model accuracy of the federated learning model is also obtained, wherein the second model accuracy is the model accuracy corresponding to the second participant. Based on the standard model accuracy of the federated learning model, the first model accuracy, and the second model accuracy, combined with the data quality index of the original sample data provided by the first participant and the data quality index of the original sample data provided by the second participant, the model contribution of the first participant to the federated learning model is calculated.

[0039] In practical collaborative modeling applications, the model contribution of each participant is influenced not only by the accuracy of the model corresponding to each participant but also by the data quality of the original sample data provided by each participant. The data quality index of the original sample data provided by the first participant may differ from that provided by the second participant. Therefore, traditional methods, which calculate the model contribution of each participant solely based on the standard model accuracy, the first model accuracy, and the second model accuracy of the federated learning model, obviously yield low accuracy. In this application, when calculating the model contribution of each participant, not only are the standard model accuracy, the first model accuracy corresponding to the first participant, and the second model accuracy corresponding to the second participant considered, but also the data quality index of the original sample data provided by the first and second participants. Therefore, combining the data quality indices of each participant allows the calculated model contribution of each participant to better reflect the degree of contribution to model training in real-world application scenarios, resulting in a more accurate final calculation of the model contribution of each participant. Attached Figure Description

[0040] Figure 1 This is a diagram illustrating the application environment of the model contribution evaluation method in one embodiment.

[0041] Figure 2 This is a flowchart illustrating a model contribution evaluation method in one embodiment;

[0042] Figure 3 for Figure 2 A flowchart illustrating the process of obtaining the first model accuracy of a federated learning model;

[0043] Figure 4 for Figure 2 A flowchart illustrating the process of obtaining the accuracy of the second model in a federated learning model;

[0044] Figure 5 This is a flowchart illustrating a method for obtaining a data quality index in one embodiment;

[0045] Figure 6 for Figure 5 A flowchart illustrating the method for generating data quality indices for each of the first original features in the first original sample data;

[0046] Figure 7 for Figure 2 A flowchart illustrating the method for calculating the model contribution of the first participant;

[0047] Figure 8 for Figure 7 A flowchart illustrating the method for calculating the model contribution of the first original feature;

[0048] Figure 9 This is a flowchart illustrating the model contribution evaluation method in a specific embodiment.

[0049] Figure 10 This is a flowchart illustrating the model contribution evaluation method in another specific embodiment;

[0050] Figure 11 This is a structural block diagram of a model contribution evaluation device in one embodiment;

[0051] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] The model contribution evaluation method provided in this application can be applied to, for example... Figure 1The application environment shown includes a first server 120 and a second server 140, with the first server 120 communicating with the second server 140 via a network. A first participant runs on the first server 120, and a second participant runs on the second server 140. The first server 120 can obtain the first model accuracy of the federated learning model, where the first model accuracy is the model accuracy corresponding to the first participant, and obtain the second model accuracy of the federated learning model, where the second model accuracy is the model accuracy corresponding to the second participant. Based on the standard model accuracy of the federated learning model, the first model accuracy, and the second model accuracy, combined with the data quality index of the original sample data provided by the first participant and the data quality index of the original sample data provided by the second participant, the first participant's contribution to the federated learning model is calculated. Here, the first server 120 and the second server 140 can be implemented using independent servers or a server cluster composed of multiple servers.

[0054] In one embodiment, such as Figure 2 As shown, a method for evaluating model contribution is provided. This embodiment illustrates the application of this method to a first server. It is understood that this method can also be applied to a terminal, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0055] Step 220: Obtain the first model accuracy of the federated learning model, wherein the first model accuracy is the model accuracy corresponding to the first participant.

[0056] The federated learning model can be a horizontal federated learning model, a vertical federated learning model, or a federated transfer learning model; this application does not limit this.

[0057] The first participant can be either the provider of the sample data or the user of the sample data; this application does not limit this.

[0058] Specifically, the first model accuracy of the federated learning model is calculated by inputting each original feature from the original sample data of the first participant and each intermediate feature from the intermediate sample data of the second participant into the federated learning model, and then calculating the first model accuracy of the federated learning model corresponding to the first participant. The intermediate sample data includes sample data obtained by perturbing the original sample data.

[0059] Step 240: Obtain the second model accuracy of the federated learning model, wherein the second model accuracy is the model accuracy corresponding to the second participant.

[0060] The second participant can be either the provider of the sample data or the user of the sample data; this application does not limit this.

[0061] Specifically, the first server receives the second model accuracy of the federated learning model sent by the second participant to the first participant. The second model accuracy is calculated by the second participant inputting each original feature from the second participant's original sample data and each intermediate feature from the first participant's intermediate sample data into the federated learning model for calculation.

[0062] Step 260: Based on the standard model accuracy, the first model accuracy, and the second model accuracy of the federated learning model, combined with the data quality index of the original sample data provided by the first participant and the data quality index of the original sample data provided by the second participant, calculate the model contribution of the first participant to the federated learning model.

[0063] The standard model accuracy of the federated learning model refers to the model accuracy calculated by inputting the sample data from the first participant and the sample data from the second participant into the federated learning model, respectively. Specifically, firstly, the sample data from the first participant and the sample data from the second participant are combined to generate real sample data; secondly, this real sample data is input into the federated learning model, and the model accuracy is calculated, thus obtaining the standard model accuracy of the federated learning model. The sample data input by the first participant into the federated learning model can be original sample data, intermediate sample data, or other sample data; this application does not limit this. Here, intermediate sample data refers to sample data obtained by perturbating the first original sample data. Similarly, the sample data input by the second participant into the federated learning model can be original sample data, intermediate sample data, or other sample data; this application does not limit this.

[0064] The data quality index refers to the data quality index of each original feature in the original sample data, representing the degree of data quality. The data quality index of the original sample data provided by the first participant can be obtained. Specifically, for each first original feature in the first original sample data, the data quality index of the first original feature is generated based on the feature validity index and / or feature diversity index of the first original feature.

[0065] Specifically, the data quality index of the original sample data provided by the second participant is obtained from the second original sample data. Specifically, for each second original feature in the second original sample data, a data quality index for the second original feature is generated based on the feature validity index and / or feature diversity index of the second original feature.

[0066] Specifically, for each original feature in the original sample data of the first participant, the first server calculates the model contribution of each original feature in the original sample data of the first participant based on the standard model accuracy of the federated learning model, the first model accuracy corresponding to the original feature, and the data quality index of the original feature; and the second model accuracy corresponding to each original feature in the original sample data of the second participant, and the data quality index of each original feature. Then, the model contributions of each original feature in the original sample data of the first participant are summed to obtain the model contribution of the original sample data of the first participant to the federated learning model, and this model contribution is taken as the model contribution of the first participant to the federated learning model.

[0067] In the aforementioned model contribution evaluation method, the model contribution of the first participant to the federated learning model is calculated based on the standard model accuracy, the first model accuracy, and the second model accuracy of the federated learning model, combined with the data quality index of the original sample data provided by the first participant and the data quality index of the original sample data provided by the second participant. However, in actual joint modeling applications, the model contribution of each participant is affected not only by the model accuracy corresponding to that participant but also by the data quality of the original sample data provided by each participant. Since the data quality index of the original sample data provided by the first participant and the data quality index of the original sample data provided by the second participant may differ, the traditional method, which calculates the model contribution of each participant solely based on the standard model accuracy, the first model accuracy, and the second model accuracy of the federated learning model, obviously yields low accuracy. In this application, when calculating the model contribution of each participant, not only the standard model accuracy, the first model accuracy corresponding to the first participant, and the second model accuracy corresponding to the second participant are considered, but also the data quality index of the original sample data provided by the first participant and the data quality index of the original sample data provided by the second participant are taken into account. Therefore, combining the data quality indices of each participant can make the calculated model contribution of each participant better reflect the degree of contribution of each participant to model training in real application scenarios, making the final calculated model contribution of each participant more accurate.

[0068] Figure 3This is a flowchart of step 220 in the above embodiment. The method for obtaining the first model accuracy of the federated learning model in this embodiment is described using an example running on a first server. Figure 3 As shown, the method for obtaining the first model accuracy of the federated learning model includes steps 222 to 226, wherein,

[0069] Step 222: Obtain the first original sample data and the first intermediate sample data of the first participant. The first intermediate sample data includes sample data obtained by perturbation processing of the first original sample data.

[0070] The original sample data provided by the first participant includes the first original sample data.

[0071] Specifically, the first original sample data is randomly distributed to obtain a randomized sample data with the same amount of data as the original sample data. This randomly distributed sample data is then perturbed to obtain the first intermediate sample data. The perturbing process is only one optional preprocessing method in this embodiment; it can also be normalization or binning, and this application does not limit it. The perturbing process refers to randomizing the original sample data, modifying sensitive data while maintaining certain features unchanged, and ensuring that the true original sample data cannot be reconstructed from the processed sample data. Normalization refers to mapping the original sample data to a certain range to remove extreme values. Binning refers to discretizing continuous features into a series of 0 / 1 discrete features, making the calculation results easier to store.

[0072] Step 224: Receive the second intermediate sample data from the second participant. The second intermediate sample data includes sample data obtained by perturbating the second original sample data from the second participant.

[0073] The second participant's original sample data provided locally on the second server includes the second original sample data. Since federated learning is a cryptographic machine learning technique, participating parties can collaboratively build the model without disclosing the underlying data or its encrypted form. Therefore, the second server does not directly send the second original sample data to the first server. Similarly, the first server does not directly send the first original sample data to the second server.

[0074] Specifically, the second participant randomly distributes the sample data in the second original sample data locally on the second server, obtaining randomly distributed sample data with the same data volume as the second original sample data. This randomly distributed sample data is then perturbed to obtain the second intermediate sample data. The second participant can then transmit the second intermediate sample data to the first participant; that is, the second server transmits the second intermediate sample data to the first server via the network. Optionally, the transmission method of the second intermediate sample data via the network can be encrypted or unencrypted; it can be parallel or serial; it can be synchronous or asynchronous; it can be simplex, half-duplex, or full-duplex data transmission; this application does not limit this.

[0075] Step 226: Input each first original feature from the first original sample data and each second intermediate feature from the second intermediate sample data into the federated learning model, and calculate the first model accuracy of the federated learning model corresponding to each first original feature.

[0076] After receiving the second intermediate sample data transmitted by the second participant, the first participant inputs the first original sample data stored locally and the second intermediate sample data received by the first participant into the federated learning model to calculate the first model accuracy of the federated learning model.

[0077] The first original sample data includes one or more first original features corresponding to the first original sample data; therefore, one or more first original features can be obtained from the first original sample data. Similarly, the second intermediate sample data also includes one or more second intermediate features corresponding to the second intermediate sample data; therefore, one or more second intermediate features can be obtained from the second intermediate sample data.

[0078] Here, the second intermediate sample data can be generated by perturbing the second original sample data. Specifically, firstly, a portion of the second original sample data is randomly selected from the second original sample data, and the selected second original sample data is perturbed to generate new sample data; secondly, the new sample data is used to replace this portion of the second original sample data to obtain the second intermediate sample data.

[0079] After generating the second intermediate sample data, one or more second intermediate features corresponding to the second intermediate sample data can be obtained from the second intermediate sample data.

[0080] Specifically, each first original feature from the first original sample data and each second intermediate feature from the second intermediate sample data are input into the federated learning model, and the first model accuracy corresponding to each first original feature is calculated. Optionally, the calculation method for the first model accuracy can be algorithm model evaluation, regression model evaluation, or classification model evaluation; this application does not limit the specific calculation method. Algorithm model evaluation refers to calculating the proportion of correctly predicted values ​​to the total number of predictions; regression model evaluation refers to calculating the difference between the model's predicted values ​​and the actual values; and classification model evaluation refers to calculating based on the confusion matrix, displaying the predicted results and actual results in a 2×2 contingency table to compare the classification results with the true information of the instances.

[0081] Assuming the first participant is an insurance company client and the second participant is a bank client, then the first participant's original sample data includes consumer behavior data and the corresponding actual insurance product recommendations. The process of inputting each first original feature from the first original sample data and each second intermediate feature from the second intermediate sample data into the federated learning model to calculate the accuracy of the first model involves inputting each first original feature from the insurance company client's consumer behavior data and each second intermediate feature from the bank client's second intermediate sample data into the federated learning model to predict insurance product recommendations and generate prediction results. The actual insurance product recommendations are then compared with the predicted results to calculate the accuracy of the first model of the federated learning model.

[0082] In this embodiment, by acquiring the first original sample data and first intermediate sample data of the first participant, and receiving the second intermediate sample data of the second participant, each first original feature in the first original sample data and each second intermediate feature in the second intermediate sample data are input into the federated learning model. The first model accuracy of the federated learning model corresponding to each first original feature is calculated. In actual joint modeling application scenarios, the model contribution of each participant is affected by the model accuracy corresponding to each participant. In traditional methods, the first participant only uses its own sample data to input into the model to obtain prediction results, and the results obtained are singular and do not have the same characteristics as the sample data of the second participant. Joint predictions often yield results with limited reliability. In this application, the first original sample data from the first participant and the second intermediate sample data from the second participant are jointly input into the federated learning model. This makes the accuracy of the resulting first model more reliable and also serves as a simulation training analysis in the subsequent calculation of the model contributions of each participant. Furthermore, by importing the data quality indices of each participant, the gap between the accuracy of the standard model and the accuracy of each participant's model can be effectively adjusted, allowing it to better reflect the overall situation in real-world application scenarios. This makes the calculated model contributions more closely resemble the actual data, thereby achieving the goal of reasonably distributing the benefits of the federated learning model among the various participants.

[0083] Figure 4 This is a flowchart of step 240 in the above embodiment. The method for obtaining the second model accuracy of the federated learning model in this embodiment is described using an example running on a second server. Figure 4 As shown, the method for obtaining the second model accuracy of the federated learning model includes steps 242 to 244, wherein,

[0084] Step 242: The first participant sends the first intermediate sample data to the second participant. The second participant inputs each second original feature in the second original sample data and each first intermediate feature in the first intermediate sample data into the federated learning model for calculation. The resulting second model accuracy of the federated learning model corresponding to each second original feature is calculated.

[0085] Specifically, the first participant sends the first intermediate sample data to the second participant. Optionally, the method of sending the first intermediate sample data can be encrypted or unencrypted, can be parallel transmission or serial transmission, can be synchronous transmission or asynchronous transmission, can be simplex data transmission, can be half-duplex data transmission, or can be full-duplex data transmission; this application does not limit this.

[0086] Specifically, each second original feature from the second original sample data and each first intermediate feature from the first intermediate sample data are input into the federated learning model, and the second model accuracy of the federated learning model corresponding to each second original feature is calculated. Optionally, the calculation method for the second model accuracy can be algorithm model evaluation, regression model evaluation, or classification model evaluation; this application does not limit the specific calculation method.

[0087] Step 244: Receive the second model accuracy of the federated learning model sent by the second participant to the first participant.

[0088] Specifically, the second participant sends the second model accuracy of the federated learning model to the first participant. Optionally, the method of sending the second model accuracy can be encrypted or unencrypted, can be parallel transmission or serial transmission, can be synchronous transmission or asynchronous transmission, can be simplex data transmission, can be half-duplex data transmission, or can be full-duplex data transmission; this application does not limit this.

[0089] In this embodiment, the first participant sends first intermediate sample data to the second participant. The second participant then inputs each second original feature from the second original sample data and each first intermediate feature from the first intermediate sample data into the federated learning model for calculation. The resulting second model accuracy of the federated learning model corresponding to each second original feature is then received from the second participant, which in turn sends the second model accuracy of the federated learning model to the first participant. This makes the model accuracy of each participant transparent and initially ensures the fairness and reasonableness of the information among them. In traditional methods, the second participant only uses its own sample data to input into the model to obtain prediction results, resulting in a single, unrelated result. Joint prediction using sample data from one participant yields prediction results with low reliability. In this application, the second original sample data from the second participant and the first intermediate sample data from the first participant are jointly input into the federated learning model, making the accuracy of the resulting second model more reliable. Simultaneously, in the subsequent calculation of the model contribution of each participant, it serves as a simulation training analysis. Furthermore, by importing the data quality index of each participant, the gap between the accuracy of the standard model and the accuracy of each participant's model can be effectively adjusted, allowing it to better reflect the overall situation in real-world application scenarios. This makes the calculated model contribution more closely resemble the real data, thereby achieving the goal of reasonably distributing the benefits of the federated learning model among the various participants.

[0090] In one embodiment, such as Figure 5As shown, a model contribution evaluation method is also provided. This embodiment uses the application of this method to a first server as an example for illustration. It can be understood that this method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0091] Step 520: Obtain the data quality index of each first original feature in the first original sample data.

[0092] Among them, the data quality index refers to the data quality index of each original feature in the original sample data, which represents the degree of good or bad data quality.

[0093] Specifically, for each original feature in the first original sample data, the first server can generate a data quality index for the original feature based on the feature validity index and / or feature diversity index of the original feature.

[0094] Step 540: Obtain the data quality index of each second original feature in the second original sample data.

[0095] Specifically, for each second original feature in the second original sample data, the second server can generate a data quality index for the second original feature based on the feature validity index and / or feature diversity index of the second original feature. The second server then sends the data quality index of the second original feature to the first server.

[0096] In this embodiment, the data quality index of each first original feature in the first original sample data is obtained, and the data quality index of each second original feature in the second original sample data is obtained. In actual joint modeling applications, the model contribution of each participant is affected not only by the accuracy of the model corresponding to each participant but also by the data quality of the original sample data provided by each participant. Furthermore, the data quality index of the original sample data provided by the first participant and the original sample data provided by the second participant may differ. Therefore, in traditional methods, the model contribution of each participant is calculated solely based on the accuracy of the standard model, the accuracy of the first model, and the accuracy of the second model in the federated learning model. Obviously, the accuracy of the obtained model contribution is low. In this application, by importing the data quality index of each participant, the gap between the accuracy of the standard model and the accuracy of each participant's model can be effectively adjusted, making it better reflect the overall situation in real application scenarios. This allows the calculated model contribution to be closer to the real data, thereby achieving the goal of reasonably distributing the benefits of the federated learning model among the participants.

[0097] Figure 6This is a flowchart of step 520 in the above embodiment. The method for generating data quality indices for each first original feature in the first original sample data in this embodiment is described using an example running on a terminal. Figure 6 As shown, steps 522 to 526 of the method for generating the data quality index of the first original feature are described, wherein...

[0098] Step 522: Obtain the feature effectiveness index of each first original feature in the first original sample data.

[0099] The feature validity index of the first original feature is determined based on the amount of valid data corresponding to one or more features in the first intermediate sample data. The amount of valid data can be the content of data in the first intermediate sample data that meets a certain range, or it can be the difference between the first intermediate sample data that meets a certain range and the real sample. This application does not limit this.

[0100] Step 524: Obtain the feature diversity index of each first original feature in the first original sample data.

[0101] The feature diversity index of the first original feature is determined based on the number of features in the first intermediate sample data. The first intermediate sample data may contain multiple sets of sample data, and the number of sample data determines the number of features of the sample data. At the same time, since there may be homogeneity or heterogeneity among the various features, it is necessary to process the features of the sample data by fusion, replacement or discretization to obtain the required feature diversity index. The specific implementation method is not limited in this application.

[0102] Step 526: For each first original feature in the first original sample data, generate a data quality index for the first original feature based on the feature validity index and / or feature diversity index of the first original feature.

[0103] Specifically, the method for generating the data quality index of the first original feature based on the feature validity index and / or the feature diversity index of the first original feature can optionally be obtained by performing a second fitting of the first original feature's feature validity index and / or the first original feature's feature diversity index using a function, or it can be obtained by multiplying the quality of each group of features of the first original feature with their corresponding feature validity index and / or feature diversity index of the first original feature and then adding them together. The specific implementation method is not limited in this application.

[0104] In this embodiment, the data quality index is influenced not only by the feature validity index but also by the feature diversity index. Therefore, if calculated solely based on either the feature validity index or the feature diversity index, as in traditional methods, the resulting data quality index would have low reliability. In this application, the data quality index comprehensively considers both the feature validity index and the feature diversity index, resulting in a higher accuracy. Therefore, by combining the data quality indices of each participant, the calculated model contribution of each participant can better reflect their contribution to model training in real-world application scenarios, making the final calculated model contribution of each participant more accurate.

[0105] Similarly, the method for generating the data quality index of the second original feature is the same as the method for generating the data quality index of the first original feature, and the detailed rendering process will not be described in detail here.

[0106] Figure 7 This is a flowchart of step 260 in the above embodiment. The method for calculating the model contribution of the first participant in this embodiment is described using an example running on a terminal. Figure 7 As shown, steps 262 to 266 of the method for calculating the model contribution of the first participant are as follows:

[0107] Step 262: For each first original feature in the first original sample data, calculate the model contribution of each first original feature based on the standard model accuracy of the federated learning model, the first model accuracy corresponding to the first original feature, the data quality index of the first original feature, the second model accuracy corresponding to each second original feature in the second original sample data, and the data quality index of each second original feature.

[0108] Specifically, for each first original feature in the first original sample data, the product of the first model accuracy corresponding to the first original feature and the data quality index of the first original feature is calculated; the first model accuracy and the first weighted sum of the data quality indices corresponding to all first original features in the first original sample data are calculated; the second model accuracy and the second weighted sum of the data quality indices corresponding to all second original features in the second original sample data are calculated; the first weighted sum and the second weighted sum are summed to generate a target weighted sum; the quotient of the product corresponding to each first original feature and the target weighted sum is calculated to generate the model contribution of each first original feature.

[0109] Step 264: Sum the model contributions of each of the first original features in the first original sample data to obtain the model contribution of the first original sample data to the federated learning model.

[0110] Specifically, the model contributions of the p first original features obtained in step 262 are summed to obtain the model contribution of the first original sample data to the federated learning model. This model contribution of the first original sample data to the federated learning model is then used as the model contribution of the first participant to the federated learning model. The formula for calculating the model contribution of the first participant to the federated learning model is shown in formula (1-1):

[0111]

[0112] Where Acc is the standard model accuracy of the federated learning model, Acc1 is the first model accuracy corresponding to the first original feature, and a is the data quality index of the first original feature.

[0113] Step 266: The contribution of the first original sample data to the federated learning model is taken as the contribution of the first participant to the federated learning model.

[0114] In this embodiment, for each first original feature in the first original sample data, the model contribution of each first original feature is calculated based on the standard model accuracy of the federated learning model, the first model accuracy corresponding to the first original feature, and the data quality index of the first original feature; and the second model accuracy corresponding to each second original feature in the second original sample data and the data quality index of each second original feature. The model contributions of each first original feature in the first original sample data are summed to obtain the model contribution of the first original sample data to the federated learning model. This model contribution of the first original sample data to the federated learning model is used as the model contribution of the first participant to the federated learning model. Summing the model contributions of all original features yields the global contribution of the first participant. Compared to traditional methods that only obtain the model contribution of a single feature before allocation, this approach is more comprehensive, making the model's returns more reasonable and convincing.

[0115] Similarly, the method for obtaining the model contribution of the second participant to the federated learning model is the same as the method for obtaining the model contribution of the first participant to the federated learning model, and the detailed rendering process will not be repeated here. The formula for calculating the model contribution of the second participant to the federated learning model is shown in formula (1-2) below:

[0116]

[0117] Where Acc is the standard model accuracy of the federated learning model, Acc2 is the second model accuracy corresponding to the second original feature, b is the data quality index of the second original feature, and q represents the model contribution of the q second original features.

[0118] Figure 8 This is a flowchart of step 262 in the above embodiment. The method for calculating the model contribution of the first original feature in this embodiment is described using an example running on a terminal. Figure 8 As shown, steps 262a to 262d of the method for calculating the model contribution of the first original feature are described, wherein...

[0119] Step 262a: For each first original feature in the first original sample data, calculate the product of the first model accuracy corresponding to the first original feature and the data quality index of the first original feature.

[0120] Specifically, the difference between the accuracy of the first model corresponding to the first original feature and the contribution of the standard model is calculated. This difference is then multiplied by the data quality index of the first original feature to obtain the product corresponding to the first original feature. The formula for calculating the product corresponding to the first original feature is shown in formula (1-3) below:

[0121] The product corresponding to the first original feature = (Acc1 - Acc) × a (Formula 1-3)

[0122] Where Acc is the standard model accuracy of the federated learning model, Acc1 is the first model accuracy corresponding to the first original feature, and a is the data quality index of the first original feature.

[0123] Step 262b: Calculate the first model accuracy corresponding to all first original features in the first original sample data and the first weighted sum of the data quality indices corresponding to all first original features.

[0124] Specifically, according to Figure 3 The accuracy of the first model obtained Figure 5 The data quality index of the first original feature obtained in step 520 and the standard model accuracy of the federated model are used to calculate the difference between the accuracy of the first model corresponding to the first original feature and the accuracy of the standard model of the federated learning model. This difference is then multiplied by the data quality index of the first original feature. Finally, the n products are summed to obtain the first weighted sum. The formula for calculating the first weighted sum is shown in formula (1-4) below:

[0125] First weighted sum = (Acc11-Acc)×a1 + (Acc12-Acc)×a2 + (Acc13-Acc)×a3 + ... + (Acc1 n -Acc)×a n Formula (1-4)

[0126] Where Acc is the standard model accuracy of the federated learning model, Acc1 is the first model accuracy corresponding to the first original feature, and a is the data quality index of the first original feature.

[0127] Step 262c: Calculate the second model accuracy corresponding to all second original features in the second original sample data, and the second weighted sum of the data quality indices corresponding to all second original features.

[0128] Specifically, according to Figure 4 The accuracy of the obtained second model Figure 5 The data quality index of the second original feature obtained in step 540 and the standard model accuracy of the federated model are used to calculate the difference between the second model accuracy corresponding to the second original feature and the standard model accuracy of the federated learning model. This difference is then multiplied by the data quality index of the second original feature. Finally, the m products are summed to obtain the second weighted sum. The formula for calculating the second weighted sum is shown in formula (1-5) below:

[0129] Second weighted sum = (Acc21-Acc)×b1 + (Acc22-Acc)×b2 + (Acc23-Acc)×b3 + ... + (Acc2 m -Acc)×b m Formula (1-5)

[0130] Where Acc is the standard model accuracy of the federated learning model, Acc2 is the second model accuracy corresponding to the second original feature, and b is the data quality index of the second original feature.

[0131] Step 262d: Summing the first weighted sum and the second weighted sum to generate the target weighted sum; calculating the quotient of the product corresponding to each first original feature and the target weighted sum to generate the model contribution of each first original feature.

[0132] Specifically, the first weighted sum and the second weighted sum obtained in steps 262b and 262c above are added together, and the sum is taken as the target weighted sum. Then, the product corresponding to the first original feature obtained in step 262a above is divided by the target weighted sum. The result is the model contribution of the first original feature. The formula for calculating the model contribution of the first original feature is shown in formula (1-6) below:

[0133]

[0134] Where Acc is the standard model accuracy of the federated learning model, Acc1 is the first model accuracy corresponding to the first original feature, and a is the data quality index of the first original feature.

[0135] In this embodiment, for each first original feature in the first original sample data, the product of the first model accuracy corresponding to the first original feature and the data quality index of the first original feature is calculated. A first weighted sum of the first model accuracy and the data quality index corresponding to all first original features in the first original sample data is then calculated. Similarly, a second weighted sum of the second model accuracy and the data quality index corresponding to all second original features in the second original sample data is calculated. The first and second weighted sums are summed to generate a target weighted sum. The quotient of the product corresponding to each first original feature and the target weighted sum is then calculated to generate the model contribution of each first original feature. In practical joint modeling applications, calculating the model contribution of each original feature allows for a more targeted understanding of the contribution of each participant to a specific feature. This enables more effective adjustments to the impact of the original feature's contribution on the contribution of each participant when subsequently calculating the model contribution of each participant based on the contribution of the original features. This better reflects the overall situation in real-world applications, making the calculated model contribution more closely resemble the actual data, thereby achieving a reasonable distribution of the benefits of the federated learning model among the participants.

[0136] Similarly, the method for obtaining the model contribution of the second original feature is the same as that for obtaining the model contribution of the first original feature, and the detailed rendering process will not be repeated here. The formula for calculating the model contribution of the second original feature is shown in formula (1-7):

[0137]

[0138] Where Acc is the standard model accuracy of the federated learning model, Acc2 is the second model accuracy corresponding to the second original feature, and b is the data quality index of the second original feature.

[0139] In a specific embodiment, such as Figure 9 As shown, a model contribution evaluation method is provided, applied to a first server, including:

[0140] Step 902: Obtain the first original sample data and the first intermediate sample data of the first participant, wherein the first intermediate sample data includes sample data obtained by perturbation processing of the first original sample data.

[0141] Combination Figure 10 As shown, the data provider, as the first participant, uses data A as the first original sample data and random data C as the first intermediate sample data. The random data C includes sample data obtained by normalizing data A. Optionally, normalization is only one preprocessing method in this embodiment; it can also be as follows: Figure 9The diagram shows perturbation processing, but it could also be bucketing processing; this application does not limit this to either.

[0142] Assume the first participant is an insurance company client, which runs on a first server. The first participant's initial sample data includes consumption behavior data on the insurance company client and the corresponding insurance product recommendations. Specifically, the first server acquires the consumption behavior data from the insurance company client and performs perturbation processing on this data to obtain processed consumption behavior data, which serves as the first intermediate sample data.

[0143] Step 904: Receive the second intermediate sample data from the second participant. The second intermediate sample data includes sample data obtained by perturbating the second original sample data from the second participant.

[0144] Combination Figure 10 As shown, the data application party, as the second participant, uses data B as the second original sample data and random data D as the second intermediate sample data. The random data D includes sample data obtained by normalizing data B. Optionally, normalization is only one preprocessing method in this embodiment; it can also be as follows: Figure 9 The diagram shows perturbation processing, but it could also be bucketing processing; this application does not limit this to either.

[0145] Assume the second participant is a bank client running on a second server. The second participant's second raw sample data includes consumption behavior data from the bank client and the corresponding insurance product recommendations. Specifically, the second server acquires the consumption behavior data from the bank client and performs perturbation processing on this data to obtain processed consumption behavior data, which serves as the second intermediate sample data.

[0146] The actual insurance products recommended based on the consumption behavior data from bank clients and the actual insurance products recommended based on the consumption behavior data from insurance company clients are the same type of insurance product.

[0147] Step 906: Input each first original feature in the first original sample data and each second intermediate feature in the second intermediate sample data into the federated learning model, and calculate the first model accuracy of the federated learning model corresponding to each first original feature.

[0148] Combination Figure 10 As shown, the data provider loads the pre-trained federated learning model and inputs each feature in data A and each feature in random data D into the federated learning model. The first model accuracy of the federated learning model corresponding to each feature in data A is calculated, and this first model accuracy is used as Acc1.

[0149] Suppose that the first server inputs each first original feature from the first original sample data and each second intermediate feature from the second intermediate sample data into the federated learning model. The process of calculating the accuracy of the first model involves inputting each first original feature from the consumer behavior data of the insurance company's clients and each second intermediate feature from the second intermediate sample data of the bank's clients into the federated learning model to predict recommended insurance products and generate insurance product prediction results. The actual recommended insurance products are compared with the predicted insurance product results to calculate the accuracy of the first model of the federated learning model.

[0150] Step 908: Receive the second model accuracy of the federated learning model sent by the second participant to the first participant;

[0151] like Figure 10 As shown, the receiving data application sends the second model accuracy of the federated learning model to the data provider, and uses this second model accuracy as Acc2. Specifically, the data requester receives random data C sent by the data provider via encrypted communication, and inputs data B and random data C into the federated learning model to obtain the second model accuracy Acc2. The data application then sends the second model accuracy Acc2 to the data provider.

[0152] Suppose that the first server receives the second model accuracy rate sent to the first server by the second server. Specifically, the second server receives first intermediate sample data sent by the insurance company client, and inputs each second original feature from the bank client's consumption behavior data and each first intermediate feature from the insurance company client's first intermediate sample data into the federated learning model to predict insurance recommendation products and generate insurance recommendation product prediction results. The actual insurance recommendation product is compared with the insurance recommendation product prediction results to calculate the second model accuracy rate of the federated learning model.

[0153] Step 910: Obtain the feature validity index of each first original feature in the first original sample data and / or obtain the feature diversity index of each first original feature in the first original sample data;

[0154] Suppose that the first server obtains the consumption behavior data of the insurance client, and determines the feature validity index based on the effective data volume of the features corresponding to the consumption behavior data characteristics, and / or obtains the feature diversity index based on some processing of the data characteristics.

[0155] Step 912: For each first original feature in the first original sample data, generate a data quality index for the first original feature based on the feature validity index and / or feature diversity index of the first original feature;

[0156] Combination Figure 10 As shown, the data provider generates a data quality index 'a'. Specifically, the data provider generates the data quality index of data A based on the feature validity index and / or feature diversity index of data A, which is the data quality index of the first original feature.

[0157] Suppose that the first server performs some kind of integration processing based on the feature validity index and / or feature diversity index of the consumer behavior data features of the insurance company's clients, and generates a data quality index of the consumer behavior data features of the insurance company's clients.

[0158] Step 914: Obtain the data quality index of each of the second original features in the second original sample data;

[0159] Combination Figure 10 As shown, the process of receiving the data quality index b sent by the data application to the data provider is the process of obtaining the data quality index of each of the second original features in the second original sample data.

[0160] Suppose that the first server receives a data quality index corresponding to the consumption behavior data of a bank client sent to the first server by the second server. Specifically, the second server obtains a feature validity index determined by the effective data volume of features corresponding to the consumption behavior data characteristics from the bank client's consumption behavior data, and / or a feature diversity index obtained by processing the data characteristics. The second server performs some integration processing based on the feature validity index and / or the feature diversity index of the bank client's consumption behavior data characteristics to generate a data quality index for the bank client's consumption behavior data characteristics. The second server then sends this data quality index to the first server.

[0161] Step 916: For each first original feature in the first original sample data, calculate the product of the first model accuracy corresponding to the first original feature and the data quality index of the first original feature;

[0162] Combination Figure 10 As shown, contribution 1 is the difference between the accuracy of the first model Acc1 and the accuracy of the standard model Acc; contribution 2 is the difference between the accuracy of the second model Acc2 and the accuracy of the standard model Acc; combining contribution 1 with the data quality index a yields the product of the accuracy of the first model and the data quality index of the first original feature.

[0163] Suppose that the first server multiplies the accuracy of the first model on the insurance company's client by the data quality index of the insurance company's client's consumer behavior data, and obtains the product corresponding to the consumer behavior data features of the insurance company's client.

[0164] Step 918: Calculate the first model accuracy corresponding to all first original features in the first original sample data and the first weighted sum of the data quality index corresponding to all first original features; calculate the second model accuracy corresponding to all second original features in the second original sample data and the second weighted sum of the data quality index corresponding to all second original features.

[0165] Suppose that the first server multiplies the accuracy of the first model corresponding to all the first original features of the insurance client with the data quality index corresponding to all the first original features, and the result is used as the first weighted sum; the first server multiplies the accuracy of the second model corresponding to all the second original features of the bank client with the data quality index corresponding to all the second original features, and the result is used as the second weighted sum.

[0166] Step 920: Summing the first weighted sum and the second weighted sum to generate the target weighted sum; calculating the quotient of the product corresponding to each first original feature and the target weighted sum to generate the model contribution of each first original feature.

[0167] Combination Figure 10 As shown, by combining contribution 1 with the data quality index a, we obtain the feature contribution corresponding to the data provider's feature. Specifically, we will... Figure 9 The first weighted sum and the second weighted sum obtained in step 918 are summed to generate the target weighted sum. The quotient of the product corresponding to each first original feature and the target weighted sum is calculated to generate the feature contribution of the data provider's feature, which is used as the model contribution of the first original feature.

[0168] Suppose that the first server adds the first weighted sum corresponding to the insurance company client and the second weighted sum corresponding to the bank client, uses the sum as the target weighted sum, divides the product corresponding to the first original feature by the target weighted sum, and uses the quotient as the model contribution of the first original feature of the insurance client.

[0169] Step 922: Sum the model contributions of each of the first original features in the first original sample data to obtain the model contribution of the first original sample data to the federated learning model.

[0170] Suppose that the first server sums the model contributions of the first original features of all insurance company clients, and uses the sum as the model contribution of the insurance company clients' consumption behavior data to the federated learning model.

[0171] Step 924: The contribution of the first original sample data to the federated learning model is taken as the contribution of the first participant to the federated learning model.

[0172] Combination Figure 10As shown, the feature contribution values ​​corresponding to each feature of all data providers are summed, and the resulting sum of feature contribution values ​​is used as the model contribution value of the first original sample data to the federated learning model. Furthermore, this sum of feature contribution values ​​is used as the model contribution value of the data providers, that is, the model contribution value of the first participant to the federated learning model.

[0173] Suppose that the first server uses the contribution of the insurance company client's consumption behavior data to the federated learning model as the model contribution of the insurance company client to the federated learning model.

[0174] In this embodiment, the process involves obtaining first original sample data and first intermediate sample data from a first participant, receiving second intermediate sample data from a second participant, inputting each first original feature from the first original sample data and each second intermediate feature from the second intermediate sample data into a federated learning model, calculating the first model accuracy of the federated learning model corresponding to each first original feature, receiving the second model accuracy of the federated learning model sent by the second participant to the first participant, obtaining the feature effectiveness index and / or feature diversity index of each first original feature in the first original sample data, and generating a data quality index for the first original features based on these indices, obtaining the data quality index of each second original feature in the second original sample data, and calculating the first original feature pair for each first original feature in the first original sample data. The product of the first model accuracy and the data quality index of the first original feature is used to calculate the first model accuracy and the first weighted sum of the data quality indices of all the first original features in the first original sample data. The second model accuracy and the second weighted sum of the data quality indices of all the second original features in the second original sample data are calculated. The first weighted sum and the second weighted sum are summed to generate a target weighted sum. The quotient of the product of each first original feature and the target weighted sum is calculated to generate the model contribution of each first original feature. The model contribution of each first original feature in the first original sample data is summed to obtain the model contribution of the first original sample data to the federated learning model. The model contribution of the first original sample data to the federated learning model is taken as the model contribution of the first participant to the federated learning model.

[0175] In practical collaborative modeling applications, the model contribution of each participant is influenced not only by the accuracy of the model corresponding to each participant but also by the data quality of the original sample data provided by each participant. The data quality index of the original sample data provided by the first participant may differ from that provided by the second participant. Therefore, traditional methods, which calculate the model contribution of each participant solely based on the standard model accuracy, the first model accuracy, and the second model accuracy of the federated learning model, obviously yield low accuracy. In this application, when calculating the model contribution of each participant, not only are the standard model accuracy, the first model accuracy corresponding to the first participant, and the second model accuracy corresponding to the second participant considered, but also the data quality index of the original sample data provided by the first and second participants. Therefore, combining the data quality indices of each participant allows the calculated model contribution of each participant to better reflect the degree of contribution to model training in real-world application scenarios, resulting in a more accurate final calculation of the model contribution of each participant.

[0176] Based on the same inventive concept, this application also provides a model contribution evaluation device for implementing the model contribution evaluation method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the model contribution evaluation device provided below can be found in the limitations of the model contribution evaluation method described above, and will not be repeated here.

[0177] In one embodiment, such as Figure 11 As shown, a model contribution evaluation device 1100 is provided, including: a first model accuracy acquisition module 1102, a second model accuracy acquisition module 1104, and a model contribution calculation module 1106, wherein:

[0178] The first model accuracy acquisition module 1102 is used to acquire the first model accuracy of the federated learning model, where the first model accuracy is the model accuracy corresponding to the first participant.

[0179] The second model accuracy acquisition module 1104 is used to acquire the second model accuracy of the federated learning model, where the second model accuracy is the model accuracy corresponding to the second participant.

[0180] The model contribution calculation module 1106 is used to calculate the model contribution of the first participant to the federated learning model based on the standard model accuracy, the first model accuracy, and the second model accuracy of the federated learning model, combined with the data quality index of the original sample data provided by the first participant and the data quality index of the original sample data provided by the second participant.

[0181] In one embodiment, the first model accuracy acquisition module 1102 includes:

[0182] The sample data acquisition unit is used to acquire the first original sample data and the first intermediate sample data of the first participant; the first intermediate sample data includes sample data obtained by perturbating the first original sample data.

[0183] The sample data receiving unit is used to receive the second intermediate sample data of the second participant; the second intermediate sample data includes sample data obtained by perturbation processing of the second original sample data of the second participant.

[0184] The computing unit is used to input each first original feature in the first original sample data and each second intermediate feature in the second intermediate sample data into the federated learning model, and to calculate the first model accuracy of the federated learning model corresponding to each first original feature.

[0185] In one embodiment, the second model accuracy acquisition module 1104 includes:

[0186] The second model accuracy receiving unit is used to receive the second model accuracy of the federated learning model sent by the second participant to the first participant. The second model accuracy is the second model accuracy of the federated learning model corresponding to each second original feature obtained by the second participant inputting each second original feature in the second original sample data and each first intermediate feature in the first intermediate sample data into the federated learning model for calculation.

[0187] In one embodiment, a model contribution evaluation device 1100 is provided, further comprising:

[0188] The first data quality index acquisition module is used to acquire the data quality index of each first original feature in the first original sample data.

[0189] The second data quality index acquisition module is used to acquire the data quality index of each second original feature in the second original sample data.

[0190] In one embodiment, the first data quality index acquisition module includes:

[0191] The feature validity index acquisition unit is used to acquire the feature validity index of each first original feature in the first original sample data.

[0192] The feature diversity index acquisition unit is used to acquire the feature diversity index of each first original feature in the first original sample data.

[0193] The data quality index generation unit is used to generate a data quality index for each first original feature in the first original sample data, based on the feature validity index and / or feature diversity index of the first original feature.

[0194] In one embodiment, the model contribution calculation module 1106 includes:

[0195] The model contribution calculation unit for the first original feature is used to calculate the model contribution of each first original feature in the first original sample data based on the standard model accuracy of the federated learning model, the first model accuracy corresponding to the first original feature and the data quality index of the first original feature, the second model accuracy corresponding to each second original feature in the second original sample data and the data quality index of each second original feature.

[0196] The first original sample data model contribution summation unit is used to sum the model contributions of each first original feature in the first original sample data to obtain the model contribution of the first original sample data to the federated learning model.

[0197] The first participant model contribution acquisition unit is used to take the model contribution of the first original sample data to the federated learning model as the model contribution of the first participant to the federated learning model.

[0198] In one embodiment, the model contribution calculation unit for the first original feature includes:

[0199] The product calculation subunit is used to calculate the product of the first model accuracy corresponding to each first original feature and the data quality index of the first original feature for each first original feature in the first original sample data.

[0200] The weighted sum calculation subunit is used to calculate the first model accuracy and the first weighted sum of the data quality index corresponding to all first original features in the first original sample data; and to calculate the second model accuracy and the second weighted sum of the data quality index corresponding to all second original features in the second original sample data.

[0201] The first original feature model contribution generation subunit is used to sum the first weighted sum and the second weighted sum to generate the target weighted sum, calculate the quotient of the product corresponding to each first original feature and the target weighted sum, and generate the model contribution of each first original feature.

[0202] Each module in the aforementioned model contribution assessment device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0203] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores task allocation data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a model contribution evaluation method.

[0204] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0205] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0206] Obtain the first model accuracy of the federated learning model, where the first model accuracy is the model accuracy corresponding to the first participant;

[0207] Obtain the second model accuracy of the federated learning model, where the second model accuracy is the model accuracy corresponding to the second participant;

[0208] The contribution of the first participant to the federated learning model is calculated based on the accuracy of the standard model, the accuracy of the first model, and the accuracy of the second model, combined with the data quality index of the original sample data provided by the first participant and the data quality index of the original sample data provided by the second participant.

[0209] In one embodiment, to obtain the first model accuracy of the federated learning model, the processor, while executing the computer program, also performs the following steps:

[0210] Obtain the first original sample data and the first intermediate sample data of the first participant; the first intermediate sample data includes sample data obtained by perturbation processing of the first original sample data;

[0211] Receive second intermediate sample data from the second participant, the second intermediate sample data including sample data obtained by perturbating the second original sample data of the second participant;

[0212] Each first original feature from the first original sample data and each second intermediate feature from the second intermediate sample data are input into the federated learning model, and the first model accuracy of the federated learning model corresponding to each first original feature is calculated.

[0213] In one embodiment, to obtain the second model accuracy of the federated learning model, the processor, while executing the computer program, also performs the following steps:

[0214] The second participant receives the second model accuracy of the federated learning model sent by the second participant to the first participant. The second model accuracy is calculated by the second participant inputting each second original feature in the second original sample data and each first intermediate feature in the first intermediate sample data into the federated learning model for calculation.

[0215] In one embodiment, the original sample data provided by the first participant includes first original sample data, and the original sample data provided by the second participant includes second original sample data. When the processor executes the computer program, it further performs the following steps:

[0216] Obtain the data quality index of each first original feature in the first original sample data;

[0217] Obtain the data quality index of each second original feature in the second original sample data.

[0218] In one embodiment, the data quality index of the first original feature includes a feature validity index and / or a feature diversity index. To obtain the data quality index of each first original feature in the first original sample data, the processor, when executing the computer program, further implements the following steps:

[0219] Obtain the feature validity index of each first original feature in the first original sample data and / or obtain the feature diversity index of each first original feature in the first original sample data;

[0220] For each of the first original features in the first original sample data, a data quality index for the first original feature is generated based on the feature validity index and / or feature diversity index of the first original feature.

[0221] In one embodiment, the contribution of the first participant to the federated learning model is calculated based on the standard model accuracy, the first model accuracy, and the second model accuracy of the federated learning model, combined with the data quality index of the original sample data provided by the first participant and the data quality index of the original sample data provided by the second participant. When the processor executes the computer program, it further implements the following steps:

[0222] For each first original feature in the first original sample data, the model contribution of each first original feature is calculated based on the standard model accuracy of the federated learning model, the first model accuracy corresponding to the first original feature, the data quality index of the first original feature, the second model accuracy corresponding to each second original feature in the second original sample data, and the data quality index of each second original feature.

[0223] The model contribution of each original feature in the first original sample data is summed to obtain the model contribution of the first original sample data to the federated learning model.

[0224] The contribution of the first original sample data to the federated learning model is taken as the contribution of the first participant to the federated learning model.

[0225] In one embodiment, for each first original feature in the first original sample data, the model contribution of each first original feature is calculated based on the standard model accuracy of the federated learning model, the first model accuracy corresponding to the first original feature, and the data quality index of the first original feature; and the second model accuracy corresponding to each second original feature in the second original sample data and the data quality index of each second original feature. When the processor executes the computer program, it further implements the following steps:

[0226] For each first original feature in the first original sample data, calculate the product of the first model accuracy corresponding to the first original feature and the data quality index of the first original feature;

[0227] Calculate the first model accuracy and the first weighted sum of the data quality indices corresponding to all first original features in the first original sample data; calculate the second model accuracy and the second weighted sum of the data quality indices corresponding to all second original features in the second original sample data;

[0228] Summing the first weighted sum and the second weighted sum generates the target weighted sum. The quotient of the product of each first original feature and the target weighted sum is calculated to generate the model contribution of each first original feature.

[0229] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0230] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0231] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0232] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0233] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0234] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A model contribution evaluation method characterized by comprising: The model includes a federated learning model including a first participant and a second participant, the method is applied to the first participant in the federated learning model, and the method includes: obtaining a first model accuracy of the federated learning model; the first model accuracy is a model accuracy corresponding to the first participant; obtaining a second model accuracy of the federated learning model; the second model accuracy is a model accuracy corresponding to the second participant; for each first original feature in the first original sample data, calculating a product corresponding to the first original feature as a product of a difference between a first model accuracy corresponding to the first original feature and a standard model accuracy and a data quality index of the first original feature; calculating a first weighted sum of differences between first model accuracies corresponding to all first original features in the first original sample data and standard model accuracies and data quality indices corresponding to the respective first original features; and calculating a second weighted sum of differences between second model accuracies corresponding to all second original features in the second original sample data and standard model accuracies and data quality indices corresponding to the respective second original features; summing the first weighted sum and the second weighted sum to generate a target weighted sum, calculating a quotient of a sum of products corresponding to each first original feature and the target weighted sum, and generating a model contribution degree of each first original feature; summing the model contribution degrees of each first original feature in the first original sample data to obtain a model contribution degree of the first original sample data to the federated learning model; taking the model contribution degree of the first original sample data to the federated learning model as a model contribution degree of the first participant to the federated learning model.

2. The method of claim 1, wherein, The method for obtaining the first model accuracy of the federated learning model includes: obtaining first original sample data and first intermediate sample data of the first participant; the first intermediate sample data includes sample data obtained by perturbing the first original sample data; receiving second intermediate sample data of the second participant; the second intermediate sample data includes sample data obtained by perturbing second original sample data of the second participant; inputting each first original feature in the first original sample data and each second intermediate feature in the second intermediate sample data into the federated learning model to calculate a first model accuracy of the federated learning model corresponding to each first original feature.

3. The method of claim 2, wherein, The method for obtaining the second model accuracy of the federated learning model includes: receiving, from the second participant, a second model accuracy of the federated learning model sent by the first participant; the second model accuracy is a second model accuracy of the federated learning model corresponding to each second original feature calculated by inputting each second original feature in the second original sample data and each first intermediate feature in the first intermediate sample data into the federated learning model by the second participant.

4. The method according to any one of claims 2-3, characterized in that, The original sample data provided by the first participant includes first original sample data, and the original sample data provided by the second participant includes second original sample data, and the method further comprises: obtaining a data quality index of each first original feature in the first original sample data; obtaining a data quality index of each second original feature in the second original sample data.

5. The method of claim 4, wherein, The data quality index of the first original feature includes a feature validity index and / or a feature diversity index; the obtaining of the data quality index of each first original feature in the first original sample data comprises: obtaining a feature validity index of each first original feature in the first original sample data; and / or obtaining a feature diversity index of each first original feature in the first original sample data; For each first original feature in the first original sample data, a data quality index of the first original feature is generated according to the feature validity index of the first original feature and / or the feature diversity index of the first original feature.

6. A model contribution degree evaluation device characterized by comprising: The model includes a federated learning model, the federated learning model includes a first participant and a second participant, the device is applied to the first participant in the federated learning model, and the device comprises: a first model accuracy obtaining module, configured to obtain a first model accuracy of the federated learning model; the first model accuracy is a model accuracy corresponding to the first participant; a second model accuracy obtaining module, configured to obtain a second model accuracy of the federated learning model; the second model accuracy is a model accuracy corresponding to the second participant; a model contribution degree calculation module, configured to calculate, for each first original feature in first original sample data, a product corresponding to the first original feature as a product of a difference between a first model accuracy corresponding to the first original feature and a standard model accuracy and a data quality index of the first original feature; calculate a first weighted sum of differences between the first model accuracy and the standard model accuracy corresponding to all first original features in the first original sample data and the data quality index corresponding to the respective first original features; calculate a second weighted sum of differences between the second model accuracy and the standard model accuracy corresponding to all second original features in the second original sample data and the data quality index corresponding to the respective second original features; sum the first weighted sum and the second weighted sum to generate a target weighted sum, calculate the quotient of the sum of the products corresponding to each first original feature and the target weighted sum to generate the model contribution degree of each first original feature; sum the model contribution degrees of each first original feature in the first original sample data to obtain the model contribution degree of the first original sample data to the federated learning model; and take the model contribution degree of the first original sample data to the federated learning model as the model contribution degree of the first participant to the federated learning model.

7. The apparatus of claim 6, wherein, The first model accuracy obtaining module comprises: The sample data acquisition unit is configured to acquire first original sample data and first intermediate sample data of the first participant, and the first intermediate sample data comprises sample data obtained by performing perturbation processing on the first original sample data. The sample data receiving unit is configured to receive second intermediate sample data of the second participant, and the second intermediate sample data comprises sample data obtained by performing perturbation processing on second original sample data of the second participant. The computing unit is configured to input each first original feature in the first original sample data and each second intermediate feature in the second intermediate sample data into the federated learning model, and calculate a first model accuracy of the federated learning model corresponding to each first original feature.

8. The apparatus of claim 6, wherein, The second model accuracy receiving unit is configured to receive a second model accuracy of the federated learning model sent by the second participant to the first participant, and the second model accuracy is a second model accuracy of the federated learning model corresponding to each second original feature, which is obtained by inputting each second original feature in the second original sample data and each first intermediate feature in the first intermediate sample data into the federated learning model and performing calculation by the second participant. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 5. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 5.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 5.

11. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 5.

Images