Data processing method and device of federated learning model and storage medium

Abstract

The application provides a data processing method and device of a federated learning model, applied to a first participant device; the method comprises: obtaining a feature set used as a training sample of a federated learning model and a target prediction result corresponding to the federated learning model of the training sample, combining each feature in the feature set to obtain a plurality of feature subsets, obtaining a weight score of each feature subset, and sampling a plurality of target feature subsets from the plurality of feature subsets based on the size of the weight score; training a linear regression model based on the plurality of target feature subsets and the prediction result corresponding to the federated learning model of the target feature subset, obtaining a model parameter when the linear regression model converges; and determining contribution information of each feature in the feature set to the target prediction result based on the model parameter. Through the application, the contribution information of each feature in a single sample can be measured, and the calculation amount of the model can be effectively reduced and the calculation efficiency can be improved.

Description

Technical Field

[0001] This application relates to artificial intelligence technology, and more particularly to a data processing method, apparatus, electronic device, and computer-readable storage medium for a federated learning model. Background Technology

[0002] With the growing trend of strengthening data privacy protection across various industries, federated learning, a technology that can collaboratively build machine learning systems using data from multiple parties while protecting data privacy, has become one of the key collaborations between enterprises and industries.

[0003] In the fields of finance and risk control, users of federated machine learning models often want to know the positive and negative impacts of each feature in a single model input on the model output. For example, for a specific sample (such as a defaulting customer), which specific feature and which values ​​of that feature have a significant impact on determining that the user is a defaulter. Additionally, it is necessary to determine the positive and negative impacts of features provided by partners on the model output. Therefore, the interpretability of federated machine learning models is particularly important.

[0004] Existing federated learning model interpretation schemes explain the model holistically by obtaining feature importance, but cannot specifically explain individual samples. Furthermore, while feature importance can reveal how many times a collaborator's feature was used, the positive or negative impact of that feature on the model's output is unknown, and determining feature contribution information involves enormous computational costs, resulting in high deployment costs. Summary of the Invention

[0005] This application provides a data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product for a federated learning model, which can measure the contribution information of each feature in a single sample and the contribution information of a second participant, while significantly reducing the computational load of the model and improving computational efficiency.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] This application provides a data processing method for a federated learning model, applied to a first participating party device, including:

[0008] Obtain the feature set of the training samples used as the federated learning model and the target prediction result of the federated learning model corresponding to the training samples, and combine each feature in the feature set to obtain multiple feature subsets. The feature set includes: features provided by a first participant with label information and features provided by at least one second participant.

[0009] Obtain the weight score of each of the feature subsets, and based on the magnitude of the weight score, sample from the multiple feature subsets to obtain multiple target feature subsets;

[0010] Based on multiple target feature subsets and the prediction results of the federated learning model corresponding to the target feature subsets, the linear regression model is trained to obtain the model parameters when the linear regression model converges.

[0011] Based on the model parameters, the contribution information of each feature in the feature set to the target prediction result is determined.

[0012] This application provides a data processing apparatus for a federated learning model, comprising:

[0013] The acquisition module is used to acquire the feature set of the training samples used as the federated learning model and the target prediction result of the federated learning model corresponding to the training samples, and to combine the features in the feature set to obtain multiple feature subsets. The feature set includes: features provided by a first participant with label information, and features provided by at least one second participant.

[0014] A sampling module is used to obtain the weight score of each of the feature subsets, and to sample multiple target feature subsets from the multiple feature subsets based on the magnitude of the weight score;

[0015] The training module is used to train a linear regression model based on multiple subsets of the target features and the prediction results of the federated learning model corresponding to the subsets of the target features, so as to obtain the model parameters when the linear regression model converges.

[0016] The determination module is used to determine the contribution information of each feature in the feature set to the target prediction result based on the model parameters.

[0017] In the above scheme, the sampling module is further configured to sort the weight scores of each feature subset in descending order of weight scores to obtain a weight score sequence; based on the weight score sequence, sequential sampling is performed starting from the feature subset with the largest weight score to obtain a first number of feature subsets as the target feature subset; wherein, the first number is less than the total number of feature subsets corresponding to the feature set.

[0018] In the above scheme, the sampling module is further used to perform regularization processing on the weight scores of each feature subset to obtain the proportional coefficient corresponding to each weight score; based on the proportional coefficient corresponding to each weight score, multiple target feature subsets are sampled from the multiple feature subsets.

[0019] In the above scheme, the sampling module is further used to sort the proportional coefficients corresponding to each weight score according to the size of the proportional coefficient, so as to obtain a proportional coefficient sequence;

[0020] Based on the order of the proportional coefficients in the proportional coefficient sequence, the following processing is performed on each proportional coefficient sequentially until the target feature subset of the target sampling number is obtained:

[0021] Obtain the current number of samples, determine the product of the scaling factor and the current number of samples, and use the product as the current capacity value;

[0022] Obtain the number of feature subsets corresponding to the scaling factor;

[0023] When the current capacity value is greater than the quantity, the feature subset corresponding to the ratio coefficient is taken as the target feature subset;

[0024] When the current capacity value is less than the number, a feature subset with the same number of samples as the current number is randomly selected from the unselected feature subset as the target feature subset.

[0025] In the above scheme, the sampling module is further configured to perform the following processing for each of the proportional coefficients:

[0026] Obtain the current number of samples, determine the product of the scaling factor and the current number of samples, and use the product as the current capacity value;

[0027] Obtain the number of feature subsets corresponding to the scaling factor;

[0028] When the current capacity value is greater than or equal to the quantity, the feature subset corresponding to the ratio coefficient is taken as the target feature subset;

[0029] When the current capacity value is less than the number, the number of feature subsets that have never been selected are randomly selected as the target feature subset.

[0030] In the above scheme, the training module is also used to obtain the transformation relationship between the target feature subset and the target training samples;

[0031] Based on the transformation relationship, the features in each of the target feature subsets are transformed to obtain the target training samples of the linear regression model;

[0032] The prediction results of the federated learning model corresponding to the target feature subset are used as the sample labels of the corresponding target training samples to train the linear regression model and obtain the model parameters when the linear regression model converges.

[0033] In the above scheme, the training module is further configured to perform the following processing for each of the target feature subsets:

[0034] The target feature subset and the feature set are compared, and the features that distinguish the target feature subset from the feature set are taken as missing features;

[0035] Each missing feature is assigned a feature value, and the missing features of the target feature subset are filled in based on the assignment results to obtain the target training samples of the linear regression model.

[0036] In the above scheme, the training module is also used to determine the default value corresponding to each missing feature; and to assign feature values ​​to the missing features based on the default values.

[0037] In the above scheme, the determining module is further used to obtain the linear mapping relationship corresponding to the linear regression model, wherein the target prediction result is the dependent variable, each feature in the feature set is the independent variable, and the model parameters are the coefficients of the independent variables;

[0038] Based on the model parameters and the linear mapping relationship, the contribution information of each feature in the feature set to the target prediction result is determined.

[0039] This application provides an electronic device, including:

[0040] Memory, used to store executable instructions;

[0041] The processor, when executing executable instructions stored in the memory, implements the data processing method of the federated learning model provided in the embodiments of this application.

[0042] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the data processing method of the federated learning model provided in this application.

[0043] This application provides a computer program product, including a computer program that, when executed by a processor, implements the data processing method of the federated learning model provided in this application.

[0044] The embodiments of this application have the following beneficial effects:

[0045] Compared with related techniques that use the feature importance output by the federated tree model to interpret the machine learning model, in this embodiment, the first participating device scores multiple feature subsets corresponding to the feature set of the training samples of the federated learning model, and then samples multiple target feature subsets based on the weight scores. A linear regression model is then trained based on the target feature subsets. The trained linear regression model can effectively measure the contribution information of each feature provided by the first participating party in the training samples used as the federated learning model, as well as the contribution information of the second participating party. At the same time, it can significantly reduce the computational load of the model and improve computational efficiency. Attached Figure Description

[0046] Figure 1 This is an optional architecture diagram of the data processing system for the federated learning model provided in this application embodiment;

[0047] Figure 2 This is an optional structural schematic diagram of the electronic device provided in an embodiment of this application;

[0048] Figure 3 This is an optional flowchart illustrating the data processing method of the federated learning model provided in the embodiments of this application;

[0049] Figure 4 This is a schematic diagram of the target feature subset sampling process provided in the embodiments of this application;

[0050] Figure 5 This is another optional schematic diagram illustrating the sampling process of the target feature subset provided in the embodiments of this application;

[0051] Figure 6 This is a schematic diagram of the training process of the linear regression model provided in the embodiments of this application;

[0052] Figure 7 This is a schematic diagram of the target feature subset conversion process provided in the embodiments of this application;

[0053] Figure 8 This is a schematic diagram of the target training samples provided in the embodiments of this application;

[0054] Figure 9 This is a schematic diagram of the feature contribution of the linear regression model provided in the embodiments of this application;

[0055] Figure 10 This is an optional flowchart of the data processing method for the federated learning model provided in the embodiments of this application;

[0056] Figure 11 This is a schematic diagram of the subset sampling process provided in an embodiment of this application;

[0057] Figure 12This is a schematic diagram illustrating the sample explanation process provided in the embodiments of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0060] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0062] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0063] 1) SHAP (SHapley Additive exPlanation) value: A model-independent interpretability analysis method based on cooperative game theory. Each prediction record has a corresponding SHAP value, and each feature also has its own corresponding SHAP value. When the SHAP value is greater than 0, it indicates that the current feature in the current sample will push the model prediction result in a positive direction, and vice versa.

[0064] 2) Median: Also known as the middle value, this is a statistical term referring to the middle number in a set of data arranged in ascending order. It represents a value in a sample, population, or probability distribution, dividing the set of values ​​into two equal parts. For a finite set of data, the median can be found by sorting all observations from highest to lowest and identifying the middle value. If there are an even number of observations, the median is usually the average of the two middle values.

[0065] 3) Mode: This refers to the value that has a clear central tendency in the statistical distribution, representing the general level of the data. It is also the value that appears most frequently in a set of data. Sometimes there are several modes in a set of data, denoted by M.

[0066] This application provides a data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product for a federated learning model, which can measure the contribution information of each feature in a single sample and the contribution information of a second participant, while significantly reducing the computational load of the model and improving computational efficiency.

[0067] Based on the above explanation of the nouns and terms used in the embodiments of this application, the data processing system of the federated learning model provided in the embodiments of this application will first be described, see [link to relevant documentation]. Figure 1 , Figure 1 This is an optional architecture diagram of the data processing system for the federated learning model provided in this application embodiment. In the data processing system 100 of the federated learning model, there is a first participant device 400 and a second participant device 410 (two second participant devices are shown for example, referred to as 410-1 and terminal 410-2 respectively, for distinction). The first participant device 400 and the second participant device 410 are interconnected through a network 300 and are also connected to the parameter aggregation device 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two, and data transmission is achieved using a wireless link.

[0068] In some embodiments, the first participant device 400 and the second participant device 410 are interconnected via network 300, and can also connect to third-party devices (collaborators, servers, etc.) that may be involved in the federated learning model via network 300.

[0069] In some embodiments, the first participating device 400 and the second participating device 410 may be laptops, tablets, desktop computers, smartphones, dedicated messaging devices, portable gaming devices, smart speakers, smartwatches, etc., or they may be client terminals of federated learning participants, such as participating devices of banks or financial institutions that store user characteristic data, but are not limited thereto. The parameter aggregation device 200 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, to assist the participating devices in federated learning to obtain a federated learning model. The network 300 may be a wide area network or a local area network, or a combination of both. The first participating device 400 and the second participating device 410 may be directly or indirectly connected through wired or wireless communication, which is not limited in this embodiment.

[0070] The first participant device 400 is used to train the federated learning model with the second participant device 410-1 and the second participant device 410-2 respectively, so as to obtain the trained federated learning model.

[0071] The first participant device 400 is also used to acquire the feature set of training samples used as the federated learning model and the target prediction results of the federated learning model corresponding to the training samples, and to combine the features in the feature set to obtain multiple feature subsets. The feature set includes: features provided by the first participant with label information and features provided by at least one second participant; to acquire the weight scores of each feature subset, and to sample multiple target feature subsets from the multiple feature subsets based on the magnitude of the weight scores; to train the linear regression model based on the multiple target feature subsets and the prediction results of the federated learning model corresponding to the target feature subsets, and to obtain the model parameters when the linear regression model converges; and to determine the contribution information of each feature in the feature set to the target prediction results based on the model parameters.

[0072] The second participating device 410 is used to send encrypted local features to the first participating device 400 to construct training samples for use as a federated learning model. When it receives a notification message sent by the first participating device 400, it fills the local features with default values ​​or target values ​​to obtain the model input corresponding to the local features.

[0073] See Figure 2 , Figure 2This is an optional structural diagram of the electronic device provided in this application embodiment. In practical applications, the electronic device 500 can be implemented as follows: Figure 1 Terminal 400 or server 200 in the middle, with electronic devices as Figure 1 Taking the server 200 shown as an example, an electronic device implementing the data processing method of the federated learning model in this application will be described. Figure 2 The illustrated electronic device 500 includes at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. The various components in the electronic device 500 are coupled together via a bus system 540. It is understood that the bus system 540 is used to implement communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 540.

[0074] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0075] User interface 530 includes one or more output devices 531 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0076] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 550 may optionally include one or more storage devices physically located away from the processor 510.

[0077] The memory 550 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory.

[0078] In some embodiments, memory 550 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0079] Operating system 551 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0080] The network communication module 552 is used to reach other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0081] Presentation module 553 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 531 (e.g., a display screen, a speaker, etc.) associated with user interface 530;

[0082] The input processing module 554 is used to detect and translate one or more user inputs or interactions from one or more input devices 532.

[0083] In some embodiments, the data processing apparatus for the federated learning model provided in this application can be implemented in software. Figure 2 A data processing apparatus 555 for a federated learning model stored in memory 550 is shown. This apparatus can be software in the form of programs and plug-ins, and includes the following software modules: an acquisition module 5551, a sampling module 5552, a training module 5553, and a determination module 5554. These modules are logically linked and can therefore be arbitrarily combined or further divided according to their implemented functions. The functions of each module will be described below.

[0084] In other embodiments, the data processing apparatus for the federated learning model provided in this application can be implemented in hardware. As an example, the data processing apparatus for the federated learning model provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the data processing method of the federated learning model provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0085] The data processing method of the federated learning model provided in this application will be described by referring to the exemplary application and implementation of the terminal provided in the embodiments of this application. This method is applied to the first participating party's device; see [link to relevant documentation]. Figure 3 , Figure 3 This is an optional flowchart illustrating the data processing method of the federated learning model provided in this application embodiment, which will be combined with... Figure 3 The steps shown are explained.

[0086] In step 101, the first participating device acquires the feature set of the training samples used as the federated learning model and the target prediction result of the federated learning model corresponding to the training samples, and combines each feature in the feature set to obtain multiple feature subsets. The feature set includes: features provided by the first participating device with label information, and features provided by at least one second participating device.

[0087] In a federated learning model, there are generally at least two participants. The first participant, also known as the active party, has label information and is denoted as Guest. The second participant is the feature provider, denoted as Host. The method provided in this application can be applied to a vertical federated learning model involving one Guest party and at least one Host party.

[0088] Taking a Guest side and a Host side as an example, in a single training sample, the Guest side and the Host side each hold a portion of the features. The Guest side provides G_1, G_2, G_3, ..., G_n, a total of n features, where n is a positive integer greater than 0; the Host side provides H_1, H_2, H_3, ..., H_m, a total of m features, where m is a positive integer greater than 0. It can be understood that the feature set of a single training sample contains a total of n+m features, {G_1, G_2, G_3, ..., G_n, H_1, H_2, H_3, ..., H_m}.

[0089] When the Guest provides n features (G_1, G_2, G_3, ..., G_n) and the Host provides m features (H_1, H_2, H_3, ..., H_m), a single training sample contains n+m features. These n+m features are combined to obtain corresponding feature subsets. The number of features in a feature subset ranges from 0 to n+m. When the number of features is 0, the feature subset is empty; when the number of features is n+m, the feature subset is the entire set. The number of feature subsets, N, is calculated using the following formula:

[0090]

[0091] In formula (1) above, n+m is the number of features. This indicates the number of feature subsets containing 0 features (i.e., the number of empty sets is 1). This represents the number of feature subsets that contain only one feature. This represents the number of feature subsets containing n+m features (i.e., the total number of features is 1).

[0092] It's important to note that in federated learning models, to protect the privacy of data from each participant, the Host's features are typically not directly transmitted to the Guest. Instead, intermediate information (such as model parameters) is transmitted between participants using encryption. Therefore, when calculating the Host's impact on the prediction result (the output of the federated learning model), the overall impact of the Host is usually determined. This involves treating all features provided by the Host as a single, unified feature, referred to as the federated feature, denoted as `host_feat`. By determining the impact of the federated feature on the prediction result and its positive or negative value, the overall impact of the corresponding Host and its positive or negative value are determined. Here, the positive or negative value refers to whether the impact is positive or negative.

[0093] When interpreting a single training sample corresponding to the federated learning model, the features provided by the Host are taken as a federated feature host_feat. The n+m features in the aforementioned single sample can be regarded as n+s features, where n is the number of features provided by the Guest and s represents the s federated features host_feat corresponding to s Hosts (i.e., one host_feat represents one Host). Here, s is a positive integer greater than 0. s=1 indicates that there is only one Host; if s=2, it indicates that there are two Hosts.

[0094] In some embodiments, interpreting a single training sample actually determines the impact of each feature in the single sample on the target prediction result, where the target prediction result refers to the prediction result obtained by the training sample through the federated learning model.

[0095] For example, in a federated learning model, a Guest party and a Host party participate in the computation. For a training sample containing n+m features, where n is the feature provided by the Guest party and m is the number of features from the Host party, to ensure data security, the m features provided by the Host party are replaced with a single host_feat feature, resulting in a total of n+1 features. These n+1 features are then combined and calculated according to the aforementioned formula (1) to obtain 2... n+1 A subset of features.

[0096] For example, a training sample S {age: 28, education: Bachelor's degree, income: 10000, occupation: IT} used as a federated learning model, whose prediction result is that the customer is a defaulter, is used. The features {age: 28, education: Bachelor's degree} are provided by the Guest, and the features {income: 10000, occupation: IT} are provided by the Host. The actual sample contains four features. In practice, to ensure the security of the Host's data, the feature {income: 10000, occupation: IT} is usually considered as a single host_feat feature. Therefore, this training sample contains two features provided by the Guest ({age: 28, education: Bachelor's degree}) and one host_feat feature, for a total of three features. The feature subset corresponding to this training sample contains 0 to 3 features, for a total of 8(2) features. 3The feature subsets are: empty set, {age: 28}, {education: Bachelor's degree}, {host_feat: {income: 10000, job: IT}}, {age: 28, education: Bachelor's degree}, {age: 28, host_feat: {income: 10000, job: IT}}, {education: Bachelor's degree, host_feat: {income: 10000, job: IT}}, {age: 28, education: Bachelor's degree, host_feat: {income: 10000, job: IT}}.

[0097] In step 102, the weight scores of each feature subset are obtained, and multiple target feature subsets are sampled from multiple feature subsets based on the magnitude of the weight scores.

[0098] Here, the feature set in each training sample used in the federated learning model corresponds to multiple feature subsets of different sizes. The subset size refers to the number of features contained in the feature subset. By scoring the subsets of different sizes, the importance of each feature can be initially estimated.

[0099] The method for scoring feature subsets is explained below. In some embodiments, the weight score of each feature subset can be determined using the following formula:

[0100] W(m)=(M-1) / (m*(Mm)) (2)

[0101] The weight score of a subset of size m is represented by W(m), where M is the number of features in a training sample used as a federated learning model, m is a positive integer greater than 0, and M > m.

[0102] Traverse the feature subsets of different sizes corresponding to the feature set, and score each feature subset according to the above formula (2). Finally, after scoring all subsets of sizes from 1 to M-1, a weight score set is obtained, which can be regarded as a scoring vector.

[0103] In some embodiments, the number of elements in the resulting scoring vector can be M-1, meaning the scoring is based on subset size. Ignoring the empty set and the entire set, a feature set containing M features corresponds to a subset size ranging from 1 to M-1. Correspondingly, the number of elements in the resulting scoring vector is also M-1, i.e., weight = [w_1, ..., w_(M-1)], where w_1 represents the weight score of the subset with size 1, and w_(M-1) represents the weight score of the subset with size M-1.

[0104] In some embodiments, ignoring the empty set and the universal set, the number of elements in the resulting scoring vector can be all feature subsets corresponding to the positive set containing M features, wherein the number of feature subsets corresponding to each subset size is represented by the following combination function.

[0105] C(r,t)=t! / r! (tr)! (3)

[0106] In the above combination function formula (3), r is an integer, t is a positive integer greater than or equal to 1, and t>r. It can be interpreted as the number of feature subsets C(r,t) of size r corresponding to the feature set containing t features, for example, C(2,5)=10.

[0107] For example, taking M=5 as an example, the number of subsets with one element, C(1,5), is 5; the number of subsets with two elements, C(2,5); the number of subsets with three elements, C(3,5); and the number of subsets with four elements, C(4,5). Ignoring the empty set and the universal set, the total number of feature subsets is: Total = C(1,5) + C(2,5) + C(3,5) + C(4,5) = 5 + 10 + 10 + 5 = 30. Using the above formula (2), feature subsets of sizes 1 to 4 are scored, i.e.:

[0108] The weight of a subset of size 1 can be calculated as: (5-1) / (1*4)=1;

[0109] The weight of a subset of size 2 can be calculated as: (5-1) / (2*3)=4 / 6;

[0110] The weight of a subset of size 3 can be calculated as: (5-1) / (3*2)=4 / 6;

[0111] The weight of a subset of size 4 can be calculated as: (5-1) / (4*1)=1.

[0112] Finally, a scoring vector, weight, is obtained. This weight can be represented in two ways: one is weight = [1, 4 / 6, 4 / 6, 1]; the other is:

[0113]

[0114] It should be noted that, according to the properties of W(m) in formula (2), the larger and smaller subsets have larger weights, and the complementary sets have equal weights.

[0115] Continuing from the previous example, taking M=5 as an example, the weight score of W(1)=W(4) is 1, and the weight score of W(2)=W(3) is 4 / 6.

[0116] After obtaining the weight scores corresponding to each feature subset, the method of sampling the target feature subset is then described. In some embodiments, the target feature subset can be obtained by sampling in the following way: sorting the weight scores of each feature subset in descending order of weight scores to obtain a weight score sequence; based on the weight score sequence, sampling is performed sequentially starting from the feature subset with the largest weight score to obtain a first number of feature subsets as the target feature subset; wherein, the first number is less than the total number of feature subsets corresponding to the feature set.

[0117] In practice, the acquired scoring vectors can be directly reordered to obtain a weight score sequence sorted from largest to smallest. Based on this weight score sequence, a first subset of target features can be directly sampled. This first subset can be a preset sampling number, and its value is less than the total number of feature subsets; it can be represented using `max_subset`.

[0118] Continuing from the previous example, taking M=5 as an example, the corresponding weight=[1,4 / 6,4 / 6,1] is reordered to get weight'=[1,1,4 / 6,4 / 6]. It can be determined that the weight scores of W(1)=W(4) are equal and higher, and the weight scores of W(2)=W(3) are equal and lower. At this time, if the preset sampling number is 10, the feature subsets with a size of 1, the number of which is C(1,5)=5, and the feature subsets with a size of 4, the number of which is C(4,5)=5, are all taken as target feature subsets, totaling 10, which is consistent with the preset sampling number. The sampling ends.

[0119] In some embodiments, a subset of target features can also be sampled by varying the magnitude of a scaling factor. See also Figure 4 , Figure 4 This is a schematic diagram of the target feature subset sampling process provided in an embodiment of this application. Based on Figure 3 Step 102 can be done Figure 4 The steps shown are implemented.

[0120] Step 1021: The first participating device performs regularization processing on the weight scores of each feature subset to obtain the proportional coefficient corresponding to each weight score.

[0121] It should be noted that the proportional coefficient is actually a regularization (normalization) operation performed on the weight scores corresponding to each feature subset obtained according to formula (2) above, resulting in a proportional coefficient with a value range between 0 and 1. The larger the proportional coefficient, the higher the corresponding weight score.

[0122] In some embodiments, based on the scoring vector weight obtained above, the proportion of the weight of each feature subset to the total weight is calculated, resulting in a proportionality coefficient. This coefficient can be combined to form a proportionality vector, denoted as p, and the calculation formula is as follows:

[0123] p=weight / (w_1+……+w_(M-1))=[p1,p2,……,p_(M-1)] (4)

[0124] For example, taking M=5 as an example, using the above formula (2), when the weight scores corresponding to each feature size are obtained as weight=[1,4 / 6,4 / 6,1], then according to the above formula (4), the weights in weight are regularized (normalized) to obtain the p-values ​​corresponding to each feature subset size:

[0125] The weights of a feature subset of size 1 are: p_1 = 1 / (1 + 4 / 6 + 4 / 6 + 1) = 0.3;

[0126] The weights of a feature subset of size 2 are: p_2 = (4 / 6) / (1+4 / 6+4 / 6+1) = 0.2;

[0127] The weights of a feature subset of size 3 are: p_3 = (4 / 6) / (1+4 / 6+4 / 6+1) = 0.2;

[0128] The weights of a feature subset of size 4 are: p_4 = 1 / (1 + 4 / 6 + 4 / 6 + 1) = 0.3;

[0129] Finally, the scaling vector p = [0.3, 0.2, 0.2, 0.3] is obtained. Furthermore, when the weight scores corresponding to the size of each feature are obtained as follows,

[0130]

[0131] The resulting set of proportionality coefficients p is represented as follows:

[0132]

[0133] Step 1022: Based on the proportional coefficients corresponding to each weight score, multiple target feature subsets are sampled from multiple feature subsets.

[0134] After obtaining the weight scores corresponding to each feature subset and performing regularization to obtain the corresponding scaling coefficients, in some embodiments, the target feature subset can be sampled in the following way: Sampling the target feature subset using scaling coefficients requires first sorting the scaling coefficients corresponding to each weight score according to their magnitude, obtaining a scaling coefficient sequence; then, based on the sorting of the scaling coefficients in the scaling coefficient sequence, performing the corresponding processing on each scaling coefficient sequentially. See also... Figure 5 , Figure 5 This is another optional schematic diagram illustrating the sampling process of the target feature subset provided in the embodiments of this application, based on Figure 4 Step 1022 can be achieved through Figure 5 The steps shown are implemented.

[0135] Step 201: The first participating device obtains the current number of samples and determines the product of the scaling factor and the current number of samples, and uses the product as the current capacity value.

[0136] Here, the current number of samples can be represented by `i_subset`, where `i` represents the index of the scaling factor in the scaling factor sequence, or it can also represent the current sampling round. If `i` starts from 1, then `1_subset` indicates that the scaling factor in the current operation is the first scaling factor in the scaling factor sequence, or the first round of sampling. It should be noted that when the scaling factor being processed is the first scaling factor in the scaling factor sequence, the current number of samples is a preset sampling threshold, represented by `max_subset`, i.e., `1_subset = max_subset`. To obtain a reasonable subset of target features, the maximum number of samples for the current sampling round is calculated in each sampling round.

[0137] In some embodiments, the maximum number of target feature subsets that can be sampled in the current sampling round is determined based on the product of the current scaling factor p_current and the current number of samples i_subset. This can also be called the capacity of the current round, denoted by capacity, i.e., capacity = i_subset * p_current.

[0138] For example, taking M=5 as an example, the sorted result is p=[p_1,p_(M-1),p_2,p_(M-2)], that is, p=[p_1,p_4,p_2,p_3]=[0.3,0.3,0.2,0.2]. In the first round of sampling, p_curren=p_1=0.3, the corresponding subset size is p_size=1, i_subset=max_subset (assumed to be 20), and a capacity number is obtained: capacity=20*0.3=6.

[0139] Step 202: Obtain the number of feature subsets corresponding to the scaling factor.

[0140] Continuing from the previous example, during the first round of sampling, the number of feature subsets corresponding to subset size p_size = 1 is C(1,5) = 5. If the conditions for the second round of sampling are met, the number of feature subsets corresponding to subset size p_size = 4 is C(4,5) = 5.

[0141] Step 203: When the current capacity value is greater than the quantity, the feature subset corresponding to the proportional coefficient is taken as the target feature subset.

[0142] Continuing from the previous example, in the first round of sampling, capacity = 6, meaning that the current round can accommodate 6 feature subsets. At this time, the number of feature subsets corresponding to subset size p_size = 1 is C(1,5) = 5. Since 5 is smaller than 6, all feature subsets with p_size = 1 are sampled as target feature subsets. The number of target feature subsets that still need to be sampled is updated to max_subset - 5 = 15. That is, when the second round of sampling is performed, i_subset = 2_subset = 15.

[0143] Step 204: When the current capacity value is less than the quantity, randomly select a feature subset with the same number of samples as the current sample number from the unselected feature subset as the target feature subset.

[0144] Continuing with the previous example, taking M=5 as an example, after the first round of sampling, the number of target feature subsets sampled is 5. A second round of sampling is needed, i.e., steps 201 to 203 are executed. It should be noted that i_subset (i.e., 2_subset) is no longer max_subset(20), but updated to 20-5=15, i.e., 2_subset=15. Therefore, the capacity obtained in step 202 is 15*0.3=4.5. Rounding 4.5 down gives 4, meaning the second round of sampling can accommodate 4 feature subsets. At this point, the number of feature subsets corresponding to subset size p_size=4 is C(4,5)=5. Since 5 is larger than 4, in the second round of sampling, not all feature subsets with p_size=4 can be sampled as feature subsets. In some embodiments, to ensure the balance of target feature subsets, the current traversal can be ended directly, and random sampling can be performed from all remaining feature subsets to obtain the remaining target feature subsets.

[0145] Continuing from the previous example, since not all feature subsets with p_size=4 can be sampled, we can randomly sample from the remaining 15 target feature subsets that need to be sampled, and finally obtain the target feature subsets of max_subset size.

[0146] For each proportional coefficient in the proportional coefficient sequence, iterate through the proportional coefficients in p in turn, and repeat steps 201 to 204 until the target feature subset of the target sampling number is obtained.

[0147] In step 103, the linear regression model is trained based on multiple target feature subsets and the prediction results of the federated learning model corresponding to the target feature subsets, so as to obtain the model parameters when the linear regression model converges.

[0148] It should be noted that some features in the target feature subset are missing (i.e. missing features). However, since the feature subset with incomplete features cannot directly obtain the prediction result through the machine learning model (the federated learning model f in this application embodiment), it is necessary to fill in the missing features and construct the model input (or training sample) of the corresponding federated learning model f.

[0149] In some embodiments, missing features in the feature set corresponding to the training samples used as federated learning models in step 101 can be determined by: determining the intersection of the target feature subset and the feature set in step 101; and determining the missing features of the target feature subset based on the obtained intersection. It is understood that missing features refer to features that distinguish the intersection from the feature set.

[0150] For example, a training sample S {age: 30, education: Master's, income: 15000, occupation: communication} is used as a federated learning model. The prediction result obtained by inputting this sample into the federated learning model represents that the customer is a defaulter. Here, the features {age: 30, education: Master's} are provided by the Guest, and the features {income: 15000, occupation: communication} are provided by the Host. The actual set of features includes 4 features. In practical applications, to ensure the security of the Host's data, the feature {income: 15000, occupation: communication} is usually considered as a single host_feat feature. Therefore, the feature set corresponding to this training sample contains 2 features provided by the Guest. We have three features: {Age: 30, Education: Master's} and one host_feat feature, corresponding to eight feature subsets (including the empty set and the full set). Specifically, these are: empty set, {Age: 30}, {Education: Master's}, {host_feat: {Income: 15000, Occupation: Communication}}, {Age: 30, Education: Master's}, {Age: 30, host_feat: {Income: 15000, Occupation: Communication}}, {Education: Master's, host_feat: {Income: 15000, Occupation: Communication}}, and {Age: 30, Education: Master's, host_feat: {Income: 15000, Occupation: Communication}}. Assuming the current target feature subset is {Age: 30, host_feat: {Income: 15000, Occupation: Communication}}, we can confirm that the missing feature {Education} is missing from this subset.

[0151] After identifying the missing features in the target feature subset, it is necessary to impute these missing features to construct the model input for the corresponding federated learning model f. Then, the prediction result for the target feature subset corresponding to f is obtained through the federated learning model. In some embodiments, missing features can be imputeed by: determining the default value corresponding to each missing feature; and assigning feature values ​​to the missing features based on the default values.

[0152] In practice, you can set the default value for missing features directly according to the actual situation, and you can use a fixed value as the default value for missing features.

[0153] Continuing from the previous example, the missing feature in the target feature subset {age: 30, host_feat: {income: 15000, occupation: communication}} is {education level}. We can pre-set the default value of the feature "education level" to "bachelor's degree" and fill the target feature subset with {education level: bachelor's degree}.

[0154] In some embodiments, the default value corresponding to the missing feature can also be determined by: obtaining a set of training samples used as a federated learning model; performing the following processing on each feature subset: obtaining the feature value corresponding to the missing feature of the feature subset in each training sample of the training sample set; determining the target value corresponding to the missing feature based on the feature value corresponding to the missing feature in each training sample; and filling the target value corresponding to the missing feature in the target feature subset.

[0155] For example, the training samples correspond to the features {income = 100, age = 20, height = 170}, and the feature subset {age = 20, height = 170}. Clearly, the missing feature is income, and the machine learning model cannot predict the samples corresponding to this feature subset. It can be filled with the mean / mode / median of "income" (which can be calculated from the training data), and the predicted value of this filled sample is used as an estimate of the model's output mean. From the training sample set, the mean of the "income" feature is calculated to be 50, the mean of the "height" feature to be 150, and the mean of the "age" feature to be 15. Therefore, the feature subset {age = 20, height = 170} can be filled with {income = 50} to obtain the corresponding filled sample {age = 20, height = 170, income = 50}.

[0156] After imputing the missing features in the target feature subset, we obtain the model input of the federated learning model f corresponding to the target feature subset. Continuing the previous example, to determine the prediction result of the target feature subset {age=20, height=170}, we input the model input of the federated learning model corresponding to {age=20, height=170, income=50} into the federated learning model f to obtain the corresponding prediction result (model output), that is, f_x({age=20, height=170}) = f({age=20, height=170, income=50}).

[0157] The prediction results of the federated learning model f corresponding to the target feature subset are used as the label data for the linear regression model h. The target feature subset is then transformed to obtain the corresponding target training samples, which are used as the training samples for the linear regression model h. The linear regression model h is then trained based on the determined label information and the training samples. For details of the training process, please refer to [link to training documentation]. Figure 6 , Figure 6 This is a schematic diagram of the training process of the linear regression model provided in the embodiments of this application, based on Figure 3 Step 103 can be achieved through Figure 6 The steps shown are implemented, combined with Figure 6 The steps shown are explained.

[0158] Step 301: The first participating device acquires the transformation relationship between the target feature subset and the target training samples.

[0159] It should be noted that the transformation relationship between the target feature subset and the target training sample refers to the use of 0 and 1 to represent each feature in the target feature subset to obtain the target training sample. That is, each feature in the training sample for the linear regression model h is represented by 0 and 1, where 1 represents the feature that actually exists in the target feature subset and 0 represents the missing feature in the target feature subset.

[0160] Step 302: Based on the transformation relationship, the features in each target feature subset are transformed to obtain the target training samples for the linear regression model.

[0161] Based on the transformation relationship obtained in step 301, the target feature subset is transformed into target training samples used as linear regression models.

[0162] In some embodiments, the specific methods for transforming features in each target feature subset can be found in [reference needed]. Figure 7 , Figure 7 This is a schematic diagram of the target feature subset conversion process provided in the embodiments of this application, based on Figure 6 Step 302 can be achieved through Figure 7 Steps 3021 to 3022 shown are implemented.

[0163] Step 3021: The first participating device compares the target feature subset and the feature set, and obtains the features that distinguish the target feature subset from the feature set as the missing features.

[0164] Here, the missing features corresponding to the target feature subset are determined by comparing it with the feature set in the training samples used as federated learning model f in step 101.

[0165] Step 3022: Assign feature values ​​to each missing feature, and fill in the missing features of the target feature subset based on the assignment results to obtain the target training samples of the linear regression model.

[0166] Here, after determining the missing features in the target feature subset, 0 is used to represent the missing features and 1 is used to represent the normal features in the target feature subset.

[0167] In some embodiments, see Figure 8 , Figure 8This is a schematic diagram of the target training samples provided in the embodiments of this application. The target training samples of the max_subset number can form a matrix of max_subset rows. Each row represents a target training sample used as a linear regression model. The features of the target training samples are represented by 0 or 1. Taking the first row as an example, suppose there is a federated learning model with a Guest party and a Host party. The Guest party provides p-1 features {X1, X2, ..., Xp-1}, and the features provided by the Host party are regarded as a feature Xp. In the target feature subset sampled from these p features, X1, X2, etc. are normal features of the target feature subset and are represented by 1, while X3, X4 are missing features of the target feature subset and are represented by 0.

[0168] Step 303: Use the prediction results of the federated learning model corresponding to the target feature subset as the sample labels of the corresponding target training samples to train the linear regression model and obtain the model parameters when the linear regression model converges.

[0169] Here, based on the target training samples determined in step 302, and the prediction results of the federated learning model f corresponding to the target feature subset, the label information (model output) of the linear regression model h is used to train the linear regression model h. The training method of the model can refer to existing model training methods, and the embodiments of this application do not limit the training method of the linear regression model.

[0170] In some embodiments, the linear regression model h can be represented by the following function:

[0171]

[0172] In this scenario, the Guest provides n features, and the feature provided by one Host is considered as one feature. k Hosts correspond to k features.

[0173] It should be noted that since the feature values ​​of the target training samples corresponding to the linear regression model are represented by 0 or 1, the linear regression model reaches convergence when the maximum number of target training samples (max_subset) have all participated in the model training. The linear regression model parameters are then obtained as follows: {w1, w2, ..., w...} n ,w h_1 ,……,w h_k}

[0174] In step 104, based on the model parameters, the contribution information of each feature in the feature set to the target prediction result is determined.

[0175] In some embodiments, the model parameters at the convergence of the linear regression model are obtained, and the obtained model parameters include the contribution information of each feature in the feature set to the target prediction result. For example, the linear mapping relationship corresponding to the linear regression model is obtained, where the target prediction result is the dependent variable, each feature in the feature set is the independent variable, and the model parameters are the coefficients of the independent variables; based on the model parameters and the linear mapping relationship, the contribution information of each feature in the feature set to the target prediction result is determined.

[0176] For example, for a 21-dimensional linear regression model,

[0177] h(x) = w1x1 + w2x2 + ... + w 20 x 20 +w h_1 x h_1

[0178] When all values ​​of x are 1, that is:

[0179] h(1) = w1×1 + w2×1 + ... + w 20 ×1+w h_1 ×1

[0180] The obtained linear regression model parameters {w1, w2, ..., w 20 ,w h_1 There are 21 weights. The first 20 weights can be directly approximated as the contribution of each feature value of the Guest side in the feature set of the training samples used as the federated learning model f in step 101. The last weight can be directly approximated as the overall contribution of the Host side.

[0181] See Figure 9 , Figure 9 This is a schematic diagram of the feature contribution of a linear regression model provided in an embodiment of this application. The training sample {age = 20, height = 170, income = 100} used in the federated learning model corresponds to a prediction result of 1.2 for the federated learning model f. When training a linear regression model, h = 0.45 + 0.4 + 0.35 = 1.2, it can be determined that the feature age = 20 contributes 0.45 to the prediction result of 1.2; the feature height = 170 contributes 0.4 to the prediction result of 1.2; and the feature income = 100 contributes 0.35 to the prediction result of 1.2.

[0182] As described above, in the federated learning estimation model provided in this application embodiment, the Guest (leading party) provides n features, and treats all features of the host party as a single federated feature host_feat, resulting in a total of n+1 features for the guest party. The guest party generates 2n+1 subsets corresponding to the n+1 features; and uses different sampling methods to select from 2...n+1 A predetermined number of target feature subsets are sampled from each feature subset. A linear regression model is trained on the Guest side using these predetermined number of subsets (e.g., from the original 2). 21 The model was trained using 2048 subsets from a subset of 2097152 (reducing computational complexity exponentially), and a weight vector was output. The first n terms of this weight vector were the estimated SHAP values ​​of each feature provided by the Guest side, and the (n+1)th term was the estimated SHAP value of the host_feat. The Guest side determined the feature importance of each feature and the total contribution of the host side based on the SHAP values ​​of the n feature values ​​of each sample and the SHAP value of the host_feat.

[0183] It should be noted that the estimated sharp values ​​for each feature in each sample obtained through the trained linear regression model are different from those obtained through actual use of 2 n+1 The sharp values ​​calculated by models trained on subsets are very close.

[0184] In this embodiment, the first participating device acquires the feature set of training samples used as a federated learning model and the target prediction results of the federated learning model corresponding to the training samples. It then combines the features in the feature set to obtain multiple feature subsets, acquires the weight scores of each feature subset, and samples multiple target feature subsets from these subsets based on the weight scores. Based on these target feature subsets and the prediction results of the federated learning model corresponding to the target feature subsets, it trains a linear regression model to obtain the model parameters at convergence. Based on the model parameters, it determines the contribution information of each feature in the feature set to the target prediction results. This application can measure the contribution information of each feature in a single sample and the contribution information of the second participating device, while significantly reducing the computational load of the model and improving computational efficiency.

[0185] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0186] The feature importance schemes provided by relevant vertical federated tree models can meet some needs. Taking a vertical federated tree scenario involving two parties (guest and host) as an example, the feature importance schemes provided by the vertical federated tree model are briefly described as follows:

[0187] (1) The guest initializes a table locally, which contains all the features of the guest and the anonymous feature IDs sent by the host. The count value of each feature is 0.

[0188] (2) Start building the decision tree. For each decision tree node, add 1 to the corresponding count value in the table, or add the split gain value.

[0189] (3) After the decision tree is built, output the feature importance table. The feature importance table can be used to interpret the model: features with high count values ​​represent those that play a high role in the modeling process.

[0190] However, the feature importance scheme provided by the above vertical federated tree model can meet the above requirements (1), but cannot meet the above requirements (2) and (3).

[0191] Firstly, regarding the aforementioned requirement (2), users cannot use feature importance to specifically interpret a single sample. For example, there is a sample S {age: 28, marital status: married, income: 10000, education: bachelor's degree, occupation: IT, place of origin: ... and other features}, which is a defaulting customer. The longitudinal federated learning model predicts the score of this sample, which is 0.1. Business users want to know how much each specific feature value in sample S (such as age = 28) contributes to the final predicted score of 0.1, and can determine whether the influence of the feature on the predicted score is positive or negative. In this way, they can combine the model with real-life experience to gain some business insights.

[0192] It's impossible to determine the impact of a specific feature value on the model output, or whether that impact is positive or negative, solely based on the feature importance score. This is because feature importance only reflects the usage of a global feature and cannot be used to analyze a specific sample.

[0193] Regarding the aforementioned requirement (3), although feature importance can reveal how many times the collaborator's features were used, the positive and negative impacts of the collaborator's features on a single sample are also unknown. If the collaborator's features can provide a significant impact on many samples, then they can be used to measure the value of the collaborator's features.

[0194] To address the problems mentioned above in the related technologies, this application provides a data processing method for federated learning models, namely, a vertical federated machine learning model interpretation method based on SHAP values. By combining the characteristics of SHAP values, this method interprets the prediction results of the vertical federated learning model in a vertical federated learning scenario and measures the overall feature value of the collaborators. Simultaneously, a subset sampling strategy is used to select multiple target subsets from the feature subsets corresponding to the feature set used as training samples for the federated learning model. These target subsets are mapped to vectors represented by 0s and 1s and used as training samples for a linear regression model. The prediction results of the federated learning model corresponding to each target subset are used as the prediction results of the linear regression model to train the linear regression model. This yields the weights of each feature when the linear regression model converges and estimates the SHAP values ​​of each feature, significantly reducing computational overhead.

[0195] First, before explaining the specific implementation process of the data processing method for the federated learning model, it is necessary to explain the relevant knowledge of SHAP.

[0196] The SHAP value is a machine learning model interpretation scheme based on the Shapely value. The Shapely value, in game theory, measures the contribution of each player (such as multiple companies). It calculates each player's contribution by considering the contributions made by each player when joining a subset of players that do not include themselves.

[0197]

[0198] Where, {x1,…,x p Let {x1, ..., x} be the set of all input features, and p be the number of all input features. p}\{x j} is excluding x j The set of all possible input features, val is an evaluation function, val(S) is the prediction of the feature subset S, |S| is the number of features in the subset S, and φ j (val) refers to the final calculated feature x. j The Shapley value (SHAP value).

[0199] For example, suppose there are companies 1, 2, and 3 that work together and make a total profit of 1.2 million. Calculate the Shapely value for each company.

[0200] 1) Enumerate all subsets: {1, 2, 3}, {1, 2}, {2, 3}, {1, 3}, {1}, {2}, {3}, and the empty set.

[0201] 2) Add each company to a subset that does not include itself, and calculate their respective marginal contributions, assuming:

[0202] N = {1, 2, 3},

[0203] v({1})=0, v({2})=0, v({3})=0,

[0204] v({1,2})=90, v({1,3})=80, v({2,3})=70,

[0205] v({1,2,3})=120

[0206] v(empty set) = 0, where v is the valuation function val; v{1} can be understood as, when only company 1 is involved, the revenue is 0 yuan; v{2,3} can be understood as, when only companies 2 and 3 cooperate, the revenue is 700,000 yuan.

[0207] For Company 1, according to the above formula (6), Company 1 is added to the subsets that do not include itself, and the marginal contribution (also known as marginal revenue) in each feature subset is calculated. The specific calculation method is as follows:

[0208] Company 1 joins subset {2, 3}, and its marginal contribution is: 2 / 6 * {v{1, 2, 3} - v{2, 3}} = 100 / 6;

[0209] Company 1 joins subset {3}, and its marginal contribution is: 1 / 6 * {v{1,3} - v{3}} = 80 / 6;

[0210] Company 1 joins subset {2}, and its marginal contribution is: 1 / 6 * {v{1,2} - v{2}} = 90 / 6;

[0211] Company 1 joins the empty set, and its marginal contribution is 0.

[0212] Therefore, the total contribution value (Shapely value) of Company 1, which generates a total profit of 1.2 million, is: 270 / 6 = 450,000.

[0213] Similarly, the above formula can be used to calculate that the contributions of companies 2 and 3 are 40W and 35W respectively.

[0214] The SHAP value in this application embodiment can be understood as a variant of the Shapely value, used to measure the contribution of a certain feature value to the prediction result (model output result), specifically:

[0215] Suppose we have a machine learning model f, input samples X, and samples containing features {1, 2, 3, ..., m}, where features are represented by subscripts i, resulting in a total of M features. N is a subset, S is a subset of features excluding i, and f_x is an estimation function that returns the average output of model f for this feature subset. The SHAP value is calculated using the same formula as the Shapely value, only with different notation, as follows:

[0216]

[0217] Continuing with the explanation of SHAP value calculation, for example, suppose we have a sample x = {age = 20, height = 170, income = 100}, and a machine learning model f predicts a score of 1.2 for this sample.

[0218] 1) Enumerate all feature subsets: {age=20, height=170, income=100}, {age=20, height=170}, {age=20, income=100}, {height=170, income=100}, {income=100}, {age=20}, {height=170}, {}.

[0219] 2) Iterate through each feature in sample x, add each feature value to a subset that does not contain itself, calculate the marginal contribution of each feature, and use the estimation function f_x to calculate the prediction result corresponding to each feature subset. Assume we have:

[0220] f_x({age=20, height=170, income=100})=1.2;

[0221] f_x({age=20, height=170})=0.9;

[0222] f_x({age=20, income=100})=0.8;

[0223] f_x({height=170, income=100})=0.7;

[0224] f_x({income = 100}) = 0;

[0225] f_x({age=20})=0;

[0226] f_x({height=170})=0;

[0227] f_x({}) = 0;

[0228] It should be noted that when the feature subset is the complete set of features in sample x, the prediction result is calculated using the estimation function f_x.

[0229] Next, the contribution of the feature value {age=20} to the predicted value of 1.2 for this sample is calculated. {age=20} is added to the feature subset that does not include itself, and the marginal contribution value is calculated according to the above formula (6), as follows:

[0230] The marginal contribution of adding {age=20} to the feature subset {height=170, income=100} is: 2 / 6*{1.2-0.7}=1 / 6;

[0231] The marginal contribution of adding {age=20} to the feature subset {income=100} is: 1 / 6*{0.8-0}=0.8 / 6;

[0232] The marginal contribution of adding {age=20} to the feature subset {age=20} is: 1 / 6*{0.9-0} = 0.9 / 6;

[0233] Adding {age=20} to the empty set results in a marginal contribution of 0.

[0234] The total contribution is 2.7 / 6 = 0.45;

[0235] Therefore, the eigenvalue {age = 20} contributed 0.45 to the final predicted score of 1.2.

[0236] Continuing with the explanation of the estimation function f_x, when given a sample and evaluating the SHAP values ​​of its various features, it is necessary to determine an estimation method f_x and use this estimation method to accurately reflect the average value of the model output (predicted score) under a certain feature subset. The following commonly used schemes are provided:

[0237] Taking the sample {age = 20, height = 170, income = 100} as an example, assuming there is a machine learning model f, the vertical federated learning model in this application embodiment is as follows:

[0238] Scheme (1): If a feature is missing in a subset, use the mean / mode / median of that feature as a substitute.

[0239] Suppose we have a feature subset {age = 20, height = 170}. Clearly, without the "income" feature, the machine learning model f cannot predict the value of this sample. In practical applications, we can fill the feature subset {age = 20, height = 170} with the mean / mode / median of the "income" feature (which can be calculated from the training data), resulting in a filled sample {age = 20, height = 170, income = T}, where T is the mean / mode / median of the income feature. The predicted value of this filled sample is then used as an estimate of the model's output average.

[0240] Suppose that from the training sample set, the average value of the "income" feature is calculated to be 50, the average value of the "height" feature is 150, and the average value of the "age" feature is 15. Then, after filling the feature subset {age = 20, height = 170}, the filled sample will be {age = 20, height = 170, income = 50}.

[0241] f_x({age=20, height=170}) = f({age=20, height=170, income=50})

[0242] For another example, suppose we have a feature subset {age = 20}. We fill this feature subset to obtain filled samples {age = 20, height = 150, income = 50}, and...

[0243] f_x({age=20}) = f({age=20, height=150, income=50}),

[0244] Additionally, if the set is empty, the output value of f_x is the average of the predicted scores of all training samples.

[0245] In scheme (2), if a feature is missing in a subset, then 0 is used directly to replace it.

[0246] For example, assuming there is a feature subset {age = 20}, the sample is padded with 0 to obtain the padded sample {age = 20, height = 0, income = 0}, and

[0247] f_x({age=20}) = f({age=20, height=0, income=0}).

[0248] Scheme (3): If a certain feature is missing in the feature subset, multiple samplings are performed within the possible range of values ​​of the missing feature to obtain multiple synthetic samples. The average of the predicted scores of the machine learning model for the multiple synthetic samples is the output value of f_x.

[0249] Assuming the age range is 10-30 and the height range is 150-180 cm.

[0250] Calculate the predicted value f_x({income=100}) for the feature subset {income=100}.

[0251] Suppose we construct 5 samples. We randomly sample the missing features (age and height) from the feature subsets, resulting in the following samples: s1 = {age = 15, height = 160, income = 100}, s2 = {age = 21, height = 164, income = 100}, s3 = {age = 29, height = 175, income = 100}, s4 = {age = 25, height = 155, income = 100}, and s5 = {age = 28, height = 180, income = 100}.

[0252] f_x({income = 100}) = [f(s1) + f(s2) + f(s3) + f(s4) + f(s5)] / 5.

[0253] Based on the explanation of the above SHAP-related knowledge, in actual federated learning scenarios, calculating the SHAP value of a feature using the above formula (6) or formula (7) often requires a very high amount of computation, which places high demands on the computing power of electronic devices (servers or terminals) and increases the complexity of actual deployment.

[0254] Generally, in practical federated learning scenarios, guests typically have one to several dozen dimensional features, while hosts usually have hundreds to thousands of dimensional features. Following the aforementioned approach, all host features are treated as a single feature, and then, using the SHAP / Shapely computation process, all subsets are enumerated. The number of subsets becomes enormous. For example, if a guest provides 20 dimensional features and a host treats all its features as a single feature, the number of subsets required to interpret any given sample is 2^32. 20+1 =2097152. If 100 samples need to be explained, then the number of samples for communication / prediction is 209715200, which is a huge amount of computation.

[0255] Based on this, a subset is sampled according to a certain strategy, and weighted linear regression is used to estimate the SHAP value, so as to greatly reduce the computational cost.

[0256] For example, a simple 0, 1 vector is used to represent the feature subset, and h is the mapping function:

[0257] h({age=20, height=170, income=100})=[1,1,1],

[0258] h({age=20, height=170}) = [1,1,0],

[0259] h({age=20, income=100})=[1,0,1],

[0260] h({height = 170, income = 100}) = [0, 1, 1],

[0261] h({income = 100}) = [1, 0, 0],

[0262] h({age=20}) = [0,1,0],

[0263] h({height = 170}) = [0, 0, 1],

[0264] h({}) = [0,0,0],

[0265] As described above, each subset will have a corresponding valuation result:

[0266] f_x({age=20, height=170, income=100})=1.2,

[0267] f_x({age=20, height=170})=0.9,

[0268] f_x({age=20, income=100})=0.8,

[0269] f_x({height=170, income=100})=0.7,

[0270] f_x({income = 100}) = 0,

[0271] f_x({age=20})=0,

[0272] f_x({height=170})=0,

[0273] f_x({})=0,

[0274] In practice, a linear regression model can be trained by using a 0 and 1 vector as X, the output of the function f_x as y, and assigning a certain weight to each vector according to the size of the subset (generally, smaller and larger subsets have higher weights). The weights of the linear regression model are the SHAP values ​​estimated for each feature.

[0275] In a federated learning scenario, the guest features and the feature importance of participating hosts are estimated using the methods described above, and the contribution values ​​are ultimately calculated.

[0276] The following section explains the specific process of estimating the SHAP value of a feature by combining the characteristics of SHAP in a vertical federated learning scenario and training a linear regression model.

[0277] To execute the data processing method (SHAP-based longitudinal federated machine learning model interpretation method) of the federated learning model provided in this application embodiment, a longitudinal federated learning model f needs to be set, and the input information of f is as follows:

[0278] The training dataset (training samples) contains N samples X0, X1, ..., Xn to be explained. The guest and host each possess a subset of features. Uppercase G_i (i is an integer) represents a feature provided by the guest (e.g., age), and uppercase H_i (i is an integer) represents a feature provided by the host. For example, the guest provides n features (G_0, G_1, ..., G_n); the host provides m features (H_0, H_1, ..., H_m). Lowercase g_j (j is an integer) represents the feature value corresponding to the feature provided by the guest (e.g., age = 20), such as g_0, g_1, ..., g_n; lowercase h_j (j is an integer) represents the feature value corresponding to the feature provided by the host, such as h_0, ..., h_m. If there are multiple hosts, add a marker at the end to distinguish each host, such as h_0_0, h_0_1, and host_0_2 representing the first feature of host_0, the first feature of host_1, and the first feature of host_2, respectively.

[0279] Estimating the setting value of function f_x: Use one of the three methods mentioned above.

[0280] Sample number sample_n: If the f_x method uses the third method mentioned above, this parameter takes effect and represents the number of times the feature values ​​are randomly sampled.

[0281] `max_subset`: A fixed value, typically 2048, representing the number of subsets sampled when estimating SHAP values ​​using linear regression.

[0282] The participants in this embodiment are: Guest (leading) party (first participant), host party (second participant), and there may be multiple host parties.

[0283] Based on the aforementioned input information, the functional modules of the system implementing the SHAP-based longitudinal federated machine learning model interpretation method provided in this application embodiment are: a subset sampling module, a sample interpretation module, and a prediction module. See also... Figure 10 , Figure 10 This is an optional flowchart of the data processing method for the federated learning model provided in the embodiments of this application, combined with... Figure 10 This section describes the overall process of data processing in federated learning models.

[0284] In step 401, the first participant counts the number of its own features and the number of second participants participating in the federated modeling.

[0285] Here, the first participant (Guest) carries label information, and the second participant (Host) is the feature provider. It is assumed that there are k (k is an integer greater than or equal to 1) Hosts participating in federated modeling.

[0286] The guest side treats all features (H_0, ..., H_m) of the host side as a single federated feature host_feat. Therefore, the guest side has n+k features: G_0, G_1, ..., G_n, host_feat_1, ..., host_feat_k.

[0287] In step 402, the first participant inputs the feature subsets of each feature in the sample to be explained into the subset sampling module, and obtains the results of a preset number of subset samplings according to the preset subset sampling strategy.

[0288] Here, the sample to be explained (i.e., a single training sample used as a federated learning model) can be regarded as containing a total of n+k features, and the final total number of samples is preset, denoted as max_subset.

[0289] The result of subset sampling can be viewed as a matrix of max_subset rows and (n+k) columns. The guest retains this matrix as the training matrix. The guest copies the first n columns of the training matrix, which is called the subset matrix, and sends the last k columns to the corresponding host. The host receives the copy and saves its own column, which becomes the subset column. The guest saves the corresponding weights, which are called the training weights.

[0290] `max_subset` represents the number of subset samples, which can be understood as the number of training samples used in the linear regression model. For example, `max_subset = 10` represents one training sample used in the machine learning model `f`, where the guest provides 4 features (n = 4), and only one host participates (k = 1). The subset sampling result is a matrix of `max_subset * (n + k)`, which is a 10x5 matrix (10 rows, 5 columns), where the last column comes from the host and the first 4 columns come from the guest.

[0291] In step 403, the first participant inputs each sample to be explained into the sample interpretation module to obtain the contribution value of each feature in the sample.

[0292] The guest side obtains the SHAP value of each sample feature and the SHAP value of host_feat from the feature contribution calculation module. The host side does not obtain any results.

[0293] The guest side adds the absolute values ​​of the SHAP values ​​of the features [G_0, G_1, ..., G_n] corresponding to all the samples to be explained X0, X1, ..., Xn, and the host_feat of each sample to be explained, to obtain the feature importance of each feature and the total contribution value of the host side.

[0294] The guest side sums the positive values ​​of the SHAP values ​​of the features [G_0, G_1, ..., G_n] corresponding to all the samples to be explained X0, X1, ..., Xn, and the positive values ​​of the SHAP value of each sample to be explained host_feat, to obtain the positive contribution of each feature.

[0295] The guest side adds the SHAP values ​​of the features [G_0, G_1, ..., G_n] corresponding to all the samples to be explained X0, X, ..., Xn, and the negative value of the SHAP value of each sample host_feat to be explained, to obtain the negative contribution of each feature.

[0296] Finally, the guest outputs the above results, while the host does not output any results.

[0297] Next, the specific functions of the subset sampling module will be explained in detail. The subset sampling module samples subsets according to a certain weighting strategy and saves the results. The module input is the total number of features, with a preset subset sampling number `max_subset`. Assume the total number of features is M, where M is a positive integer greater than 1. See also... Figure 11 , Figure 11 This is a schematic diagram of the subset sampling process provided in the embodiments of this application, based on Figure 10 Step 402 can be achieved through Figure 11 The specific steps shown are implemented. Combined with... Figure 11 The steps shown illustrate the subset sampling process.

[0298] Step 501: The first participating device enumerates the size of all possible feature subsets.

[0299] From 1 to M-1 (ignoring the empty set and the universal set), taking M=5 as an example, the number of subsets with one element is C(1,5), the number of subsets with two elements is C(2,5), the number of subsets with three elements is C(3,5), and the number of subsets with four elements is C(4,5). The total number of subsets is: Total = C(1,5) + C(2,5) + C(3,5) + C(4,5) = 5 + 10 + 10 + 5 = 30.

[0300] Step 502: Weight scores are assigned to subsets of different sizes to obtain a scoring vector, where different sizes refer to the number of elements in the subset.

[0301] The subsets are weighted using the aforementioned weighting formula (2). Finally, all feature subsets of sizes from 1 to M-1 are scored, resulting in a weighted vector:

[0302] weight = [w_1, ..., w_(M-1)], where w_1 represents the weight score of a subset with a size of 1.

[0303] Taking M=5 as an example, the above formula (2) is used to score all subsets of sizes from 1 to 4: the scoring vector weight = [1, 4 / 6, 4 / 6, 1] is obtained.

[0304] Step 503: Calculate the proportion of the weight of each feature subset to the whole based on the scoring vector to obtain the proportion vector.

[0305] Here, the proportion vector is denoted as p, and the proportion vector of the corresponding weight vector is obtained by using the aforementioned formula (4), and the value range of each element in the proportion vector is from 0 to 1.

[0306] For example, taking M=5 as an example, through step 503, we obtain weight=[1,4 / 6,4 / 6,1], and thus obtain the p value corresponding to weight: p_1=1 / (1+4 / 6+4 / 6+1)=0.3, p_2=1 / 5=0.2, p_3=0.2, p_4=0.3, and obtain the scale vector p=[0.3,0.2,0.2,0.3].

[0307] Step 504: Sort the proportional vectors from largest to smallest to obtain the sorted proportional vectors.

[0308] From the properties of W(m) in formula (2), we know that the larger and smaller subsets have larger weights, and the complementary sets have equal weights.

[0309] For example, with M=5, W(1)=W(4), W(2)=W(3), so the sorted result is p=[p_1,p_(M-1),p_2,p_(M-2)], that is, p=[p_1,p_4,p_2,p_3]=[0.3,0.3,0.2,0.2].

[0310] Step 505: Determine the capacity corresponding to the current sampling round.

[0311] It should be noted that the proportional coefficient corresponding to the current round is denoted as p_max, and the size of the corresponding subset is p_size. p_max is multiplied by the total number of samples to be sampled in the current round, i_subset, where i represents the sampling round. When i=1, i is the first round of sampling, i_subset=max_subset, resulting in a capacity: capacity=i_subset*p_max.

[0312] For example, with M=5, and max_subset=20, the sorted proportion vector is: p=[p_1,p_4,p_2,p_3]=[0.3,0.3,0.2,0.2]. If we take the first round of sampling as an example, p_max=p_1, p_size=1, and we get a capacity of capacity=0.3*20=6.

[0313] Step 506: Compare whether the current capacity is greater than or equal to the number of feature subsets corresponding to the current round. If yes, proceed to step 507; otherwise, proceed to step 509.

[0314] Here, the capacity obtained in step 505 is compared with the size of C(p_size,M), and different operations are performed based on the comparison result.

[0315] Step 507: Add the vector representation of the feature subset of the current sampling round to the vector result.

[0316] For example, taking M=5 as an example, in the first round of sampling, when p_size=1, the vectors [1,0,0,0,0], [0,1,0,0,0], [0,0,1,0,0], [0,0,0,1,0], and [0,0,0,0,1] are added to the vector result; the weight of each vector in the linear regression model is calculated using the formula: w_(p_size) / C(p_size,M).

[0317] Let M = 5, p_size = 1, and the subset size be 1. The weights in the linear regression model are w_1 / C(1,5) = 1 / 5. Add the weights corresponding to each vector to the weight list w_list. Update the value of i_subset to: i_subset = max_subset - C(p_size,M) for the next round of subset selection.

[0318] Step 508: Update the scoring vector and determine whether the elements in the scoring vector are empty. If so, proceed to step 509; otherwise, proceed to step 503.

[0319] Here, w_(p_size) is removed from the aforementioned weight vector list. If the weight is ultimately empty, proceed to step 509; otherwise, return to step 503. That is, the process is repeated to start the next round of subset selection.

[0320] Step 509: When the remaining current sample number max_subset cannot accommodate the complete subset, random sampling is performed on the remaining current sample number max_subset.

[0321] Here, the current sample size is the updated max_subset, i.e., i_subset. Calculate p for the remaining weights in the weights, where p is the probability of each size subset being selected. Then, randomly sample max_subset subsets, and randomly generate a 0, 1 vector based on the subset size. Add this vector to the result, and the corresponding weight is sum(weight) / max_subset, i.e., the remaining weights divided by the remaining sample size. Proceed to step 507.

[0322] Step 510: All subsets have been sampled. Return the vector results and the corresponding weight list.

[0323] For example, see Figure 8 The vector results shown should be understood as follows: the vector results can be viewed as a vector matrix composed of 0s and 1s. Each row is obtained by mapping the sampled feature subset. The mapping rule is that missing features in the subset are represented by 0s, and normal features in the subset are represented by 1s. The total number of rows in the vector matrix represents the total number of sampled samples.

[0324] Next, the function of the sample interpretation module will be explained. This module takes a sample X to be interpreted as input. (See also...) Figure 12 , Figure 12 This is a schematic diagram illustrating the sample explanation process provided in the embodiments of this application, based on Figure 10 Step 403 can be achieved through Figure 12 The specific steps shown are implemented. Combined with... Figure 12 The steps shown illustrate the sample interpretation process.

[0325] Step 601: The first participant and the second participant each input the feature part of the sample to be explained held into the prediction module to obtain a prediction value as a baseline value.

[0326] Here, the sample X to be explained is the sample used in the federated learning model. Looking at the settings of the f_x method, if it is Method 1 as described above, the guest / host calculates the mean, median, or mode of each feature based on the training data as filler values; if it is Method 2 as described above, 0 is used as filler values; if it is Method 3 as described above, the guest and host count the possible values ​​of each feature for random sampling.

[0327] Step 602: The first participant loads its own subset matrix, and the second participant loads its own subset columns, generating the same number of samples to be predicted for use in the federated learning model.

[0328] Here, the guest generates a corresponding subset based on the 0 and 1 feature vectors of each row in the subset matrix, and then generates the sample to be predicted for f_x based on the guest features of the subset and X. The host generates the sample to be predicted for f_x based on each row in the subset column.

[0329] The rule for generating the samples to be predicted for use in the federated learning model is as follows: if the value is 0, the host generates a new sample using the corresponding f_x scheme; otherwise, if the value is 1, the host simply copies the host features of X as a new sample.

[0330] At this point, the guest and multiple hosts will generate the same number of corresponding samples to be predicted for f_x.

[0331] Step 603: The first participant and the second participant respectively input the samples to be predicted generated by themselves into the prediction module, execute federated prediction, and obtain the prediction results of the federated learning model corresponding to each sample to be predicted.

[0332] Here, the prediction result of the federated learning model corresponding to the sample can also be called the sample's score vector, which is y, where each item in y is subtracted from the baseline value.

[0333] Step 604: The first participant uses the training matrix as samples, the training weights as weights, and the predicted values ​​of the corresponding federated learning model of the sample to be predicted as labels to train a linear regression model.

[0334] Here, the predicted value y of the corresponding federated learning model for the sample to be predicted is used as the label of the linear regression model. The trained linear regression model can be untrimmed (i.e., biased). It should be noted that the first n weights of the linear regression model correspond to the contribution of the guest features, and the last k weights correspond to the contributions of each host. The weights are extracted and returned as the interpretation result.

[0335] Finally, the prediction module is explained. The prediction module takes specific samples as input. The guest and host load the longitudinal federated machine learning model f, predict the input samples based on f, and return the predicted values.

[0336] The optimized longitudinal federated learning model interpretation method provided in this application can determine the feature contribution of each feature provided by the Guest in a single sample, and measure the feature contribution of the Host. The same applies to the case of multiple hosts, thus supplementing the missing features in the federated scenario. In addition, it retains the same measurement of global feature importance as traditional methods. The optimization of subset sampling greatly reduces the complexity of communication and computation, making the functions of interpreting multiple samples and measuring contributions in the federated scenario practical.

[0337] The following description continues to illustrate the exemplary structure of the data processing device 555 for the federated learning model provided in this application embodiment as a software module. In some embodiments, such as... Figure 2 As shown, the software modules in the data processing device 555 storing the federated learning model in memory 540 may include:

[0338] The acquisition module 5551 is used to acquire the feature set of the training samples used as the federated learning model and the target prediction result of the federated learning model corresponding to the training samples, and to combine each feature in the feature set to obtain multiple feature subsets. The feature set includes: features provided by a first participant with label information, and features provided by at least one second participant.

[0339] The sampling module 5552 is used to obtain the weight scores of each feature subset and to sample multiple target feature subsets from multiple feature subsets based on the magnitude of the weight scores.

[0340] Training module 5553 is used to train a linear regression model based on multiple target feature subsets and the prediction results of the federated learning model corresponding to the target feature subsets, so as to obtain the model parameters when the linear regression model converges.

[0341] The determination module 5554 is used to determine the contribution information of each feature in the feature set to the target prediction result based on the model parameters.

[0342] In some embodiments, the sampling module 5552 is further configured to sort the weight scores of each feature subset in descending order of weight scores to obtain a weight score sequence; based on the weight score sequence, sequentially sample from the feature subset with the largest weight score to obtain a first number of feature subsets as the target feature subset; wherein, the first number is less than the total number of feature subsets corresponding to the feature set.

[0343] In some embodiments, the sampling module 5552 is further configured to perform regularization processing on the weight scores of each feature subset to obtain the proportional coefficients corresponding to each weight score; and to obtain multiple target feature subsets by sampling from multiple feature subsets based on the proportional coefficients corresponding to each weight score.

[0344] In some embodiments, the sampling module 5552 is further configured to sort the proportional coefficients corresponding to each weight score according to the size of the proportional coefficient, to obtain a proportional coefficient sequence; and according to the sorting of the proportional coefficients in the proportional coefficient sequence, perform the following processing on each proportional coefficient in sequence until the target feature subset of the target sampling number is obtained: obtain the current sampling number, and determine the product of the proportional coefficient and the current sampling number, and use the product as the current capacity value; obtain the number of feature subsets corresponding to the proportional coefficient; when the current capacity value is greater than the number, use the feature subset corresponding to the proportional coefficient as the target feature subset; when the current capacity value is less than the number, randomly select the same number of feature subsets as the current sampling number from the unselected feature subsets as the target feature subset.

[0345] In some embodiments, the sampling module 5552 is further configured to perform the following processing for each scaling factor: obtain the current number of samples, determine the product of the scaling factor and the current number of samples, and use the product as the current capacity value; obtain the number of feature subsets corresponding to the scaling factor; when the current capacity value is greater than or equal to the number, use the feature subset corresponding to the scaling factor as the target feature subset. When the current capacity value is less than the number, randomly select a number of feature subsets from the unselected feature subsets as the target feature subset.

[0346] In some embodiments, the training module 5553 is further configured to obtain the transformation relationship between the target feature subset and the target training samples; based on the transformation relationship, transform the features in the target feature subset to obtain the target training samples of the linear regression model; use the prediction results of the federated learning model corresponding to the target feature subset as the sample labels of the corresponding target training samples to train the linear regression model and obtain the model parameters when the linear regression model converges.

[0347] In some embodiments, the training module 5553 is further configured to perform the following processing for each target feature subset: compare the target feature subset with the feature set, and obtain the features that distinguish the target feature subset from the feature set as missing features; assign feature values ​​to each missing feature, and fill in the missing features of the target feature subset based on the assignment results to obtain the target training samples of the linear regression model.

[0348] In some embodiments, the training module 5553 is further configured to determine the default value corresponding to each missing feature; and assign feature values ​​to the missing features based on the default values.

[0349] In some embodiments, the determining module 5554 is further configured to obtain the linear mapping relationship corresponding to the linear regression model, wherein the target prediction result is the dependent variable, each feature in the feature set is the independent variable, and the model parameters are the coefficients of the independent variables; based on the model parameters and the linear mapping relationship, the contribution information of each feature in the feature set corresponding to the target prediction result is determined.

[0350] It should be noted that the description of the apparatus in this application embodiment is similar to the description of the method embodiment above, and has similar beneficial effects as the method embodiment, so it will not be repeated.

[0351] This application provides a computer program product, including a computer program, characterized in that, when executed by a processor, the computer program implements the data processing method of the federated learning model provided in this application.

[0352] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to perform the method provided in this application, for example... Figure 3 The data processing method of the federated learning model is shown.

[0353] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0354] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0355] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0356] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0357] In summary, the embodiments of this application not only expand the applicability in federated learning scenarios, but also accurately measure the contribution information of each feature in a single training sample as well as the contribution information of the second participant, while significantly reducing the computational load of the model and improving computational efficiency.

[0358] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A data processing method for a federated learning model, characterized in that, Applied to the equipment of the first participant: The feature set of training samples used as a federated learning model and the target prediction result of the federated learning model corresponding to the training samples are obtained. The features in the feature set are combined to obtain multiple feature subsets. The feature set includes: features provided by a first participant with label information and features provided by at least one second participant; wherein, the features provided by the first participant include age and education level; and the features provided by the second participant include income and occupation. Obtain the weight score of each of the feature subsets, and based on the magnitude of the weight score, sample from the multiple feature subsets to obtain multiple target feature subsets; Based on multiple target feature subsets and the prediction results of the federated learning model corresponding to the target feature subsets, the linear regression model is trained to obtain the model parameters when the linear regression model converges. Based on the model parameters, determine the contribution information of each feature in the feature set to the target prediction result; Based on the magnitude of the weight scores, multiple target feature subsets are sampled from the multiple feature subsets, including: The weight scores of each of the aforementioned feature subsets are regularized to obtain the proportional coefficients corresponding to each weight score; According to the size of the proportional coefficient, the proportional coefficients corresponding to each weight score are sorted to obtain the proportional coefficient sequence; Based on the order of the proportional coefficients in the proportional coefficient sequence, the following processing is performed on each proportional coefficient sequentially until the target feature subset of the target sampling number is obtained: Obtain the current number of samples, determine the product of the scaling factor and the current number of samples, and use the product as the current capacity value; Obtain the number of feature subsets corresponding to the scaling factor; When the current capacity value is greater than the quantity, the feature subset corresponding to the ratio coefficient is taken as the target feature subset; When the current capacity value is less than the number, a feature subset with the same number of samples as the current number is randomly selected from the unselected feature subset as the target feature subset.

2. The method according to claim 1, characterized in that, The process of training a linear regression model based on multiple subsets of target features and the prediction results of the federated learning model corresponding to the subsets of target features to obtain the model parameters when the linear regression model converges includes: Obtain the transformation relationship between the target feature subset and the target training samples; Based on the transformation relationship, the features in each of the target feature subsets are transformed to obtain the target training samples of the linear regression model; The prediction results of the federated learning model corresponding to the target feature subset are used as the sample labels of the corresponding target training samples to train the linear regression model and obtain the model parameters when the linear regression model converges.

3. The method according to claim 2, characterized in that, The transformation of features in each of the target feature subsets based on the transformation relationship to obtain the target training samples for the linear regression model includes: The following processing is performed on each of the target feature subsets: The target feature subset and the feature set are compared, and the features that distinguish the target feature subset from the feature set are taken as missing features; Each missing feature is assigned a feature value, and the missing features of the target feature subset are filled in based on the assignment results to obtain the target training samples of the linear regression model.

4. The method according to claim 3, characterized in that, The step of assigning feature values ​​to each of the missing features includes: Determine the default value corresponding to each of the missing features; Based on the default value, feature values ​​are assigned to the missing features.

5. The method according to claim 1, characterized in that, The step of determining the contribution information of each feature in the feature set to the target prediction result based on the model parameters includes: Obtain the linear mapping relationship corresponding to the linear regression model, wherein the target prediction result is the dependent variable, each feature in the feature set is the independent variable, and the model parameters are the coefficients of the independent variables; Based on the model parameters and the linear mapping relationship, the contribution information of each feature in the feature set to the target prediction result is determined.

6. A data processing apparatus for a federated learning model, characterized in that, include: The acquisition module is used to acquire a feature set of training samples used as a federated learning model and the target prediction result of the federated learning model corresponding to the training samples, and to combine the features in the feature set to obtain multiple feature subsets. The feature set includes: features provided by a first participant with label information, and features provided by at least one second participant; wherein, the features provided by the first participant include age and education level; and the features provided by the second participant include income and occupation. A sampling module is used to obtain the weight score of each of the feature subsets, and to sample multiple target feature subsets from the multiple feature subsets based on the magnitude of the weight score; The training module is used to train a linear regression model based on multiple subsets of the target features and the prediction results of the federated learning model corresponding to the subsets of the target features, so as to obtain the model parameters when the linear regression model converges. The determination module is used to determine the contribution information of each feature in the feature set to the target prediction result based on the model parameters. The sampling module is further configured to perform regularization processing on the weight scores of each feature subset to obtain the proportional coefficients corresponding to each weight score; sort the proportional coefficients corresponding to each weight score according to their magnitude to obtain a proportional coefficient sequence; and perform the following processing on each proportional coefficient in sequence according to the sorting of the proportional coefficients until the target feature subset with the target sampling number is obtained: obtain the current sampling number and determine the product of the proportional coefficient and the current sampling number, and use the product as the current capacity value; obtain the number of feature subsets corresponding to the proportional coefficient; when the current capacity value is greater than the number, use the feature subset corresponding to the proportional coefficient as the target feature subset; when the current capacity value is less than the number, randomly select the same number of feature subsets as the current sampling number from the unselected feature subsets as the target feature subset.

7. An electronic device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the data processing method of the federated learning model according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores executable instructions for implementing the data processing method of the federated learning model according to any one of claims 1 to 5 when executed by a processor.

9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the data processing method of the federated learning model as described in any one of claims 1 to 5.

Images