Data processing method, device and storage medium of federated learning model

Abstract

The present application provides a data processing method for a federated learning model, which is applied to a first participant device; the method includes: obtaining at least one first feature provided by the first participant device for training the federated learning model; receiving at least one anonymous feature sent by the second participant device, the anonymous feature corresponding to the second feature used for training the federated learning model; selecting at least one feature from the first feature and the anonymous feature, and obtaining the prediction results of the federated learning model corresponding to each feature; constructing a training sample of a linear model based on each feature, and training the linear model based on the training sample and the prediction results to obtain the model parameters when the linear model converges; determining the first contribution information corresponding to each first feature and the second contribution information corresponding to each anonymous feature based on the model parameters. Through the present application, the contribution information of each feature in a single sample can be measured, and at the same time, the calculation amount of the model can be effectively reduced, and the calculation efficiency can be improved.

Description

Technical Field

[0001] The present application relates to artificial intelligence technology, and in particular to a data processing method, device, electronic device and computer-readable storage medium for a federated learning model. Background Art

[0002] As all industries gradually strengthen data privacy protection, federated learning, a technology that can coordinate multi-party data to establish machine learning while protecting data privacy, has become one of the keys to cooperation between enterprises and industries.

[0003] In the fields of finance and risk control, users of federated machine learning models often want to know the positive and negative impact of each feature in a single model input on the model output. For example, for a specific sample (for example, a defaulting customer), which specific feature and which values ​​of the feature have an important impact on determining that the user is a defaulting user. In addition, it is also necessary to determine the positive and negative impact of the features provided by the partner on the model output. Therefore, the interpretability of federated machine learning models is particularly important.

[0004] The related federated learning model interpretation scheme interprets the model as a whole by obtaining the feature importance, but cannot specifically explain the feature importance of each feature in a single sample. In addition, the model calculation amount is very large when determining the feature importance, and the actual deployment cost is high. Summary of the invention

[0005] The embodiments of the present application provide a data processing method, device, electronic device, computer-readable storage medium and computer program product of a federated learning model, which can measure the contribution information of each feature in a single sample, and can effectively reduce the calculation amount of the model and improve the calculation efficiency.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] The embodiment of the present application provides a data processing method of a federated learning model, which is applied to a first participant device, including:

[0008] Acquire at least one first feature provided by the first participant device for training the federated learning model;

[0009] Receiving at least one anonymous feature sent by a second participant device, wherein the anonymous feature corresponds to a second feature used to train the federated learning model;

[0010] Selecting at least one feature from the first feature and the anonymous feature, and obtaining a prediction result of the federated learning model corresponding to each feature;

[0011] Based on the at least one feature, construct a training sample of the linear model, and based on the training sample and the prediction result, train the linear model to obtain model parameters when the linear model converges;

[0012] Based on the model parameters, first contribution information of each of the first features corresponding to the prediction result and second contribution information of each of the anonymous features corresponding to the prediction result are determined.

[0013] The present application embodiment provides a data processing device for a federated learning model, including:

[0014] An acquisition module, configured to acquire at least one first feature provided by the first participant device for training the federated learning model;

[0015] A receiving module, configured to receive at least one anonymous feature sent by a second participant device, wherein the anonymous feature corresponds to a second feature used to train the federated learning model;

[0016] A selection module, used to select at least one feature from the first feature and the anonymous feature, and obtain a prediction result of each feature corresponding to the federated learning model;

[0017] A training module, configured to construct a training sample of a linear model based on the at least one feature, and to train the linear model based on the training sample and the prediction result to obtain a model parameter when the linear model converges;

[0018] A determination module is used to determine, based on the model parameters, first contribution information of each of the first features corresponding to the prediction result, and second contribution information of each of the anonymous features corresponding to the prediction result.

[0019] In the above solution, the receiving module is further used to send an anonymous feature acquisition request carrying a sample identifier to the second participant device;

[0020] The sample identifier is used to identify the sample to which the first feature belongs for training the federated learning model, and the anonymous feature acquisition request is used for the second participant device to respond to the anonymous feature acquisition request to determine the second feature corresponding to the sample identifier and the anonymous feature corresponding to the second feature;

[0021] An anonymous feature corresponding to the sample identifier is received and returned by the second participant device.

[0022] In the above solution, the receiving module is used to sum the first contribution information of each of the first features corresponding to the prediction result and the second contribution information of each of the anonymous features corresponding to the prediction result to obtain the summary contribution information;

[0023] The ratio of each second contribution information to the aggregate contribution information is obtained respectively, and the ratio is sent to the second participant device, so that the second participant device determines the contribution information of the second feature corresponding to each anonymous feature based on the ratio.

[0024] In the above solution, the selection module is further used to construct a feature set including the at least one first feature and the at least one anonymous feature;

[0025] Acquire a feature subset included in the feature set;

[0026] A target number of target feature subsets are obtained by sampling from the plurality of feature subsets, and features included in the target feature subsets are used as the at least one feature selected.

[0027] In the above solution, the selection module is also used to obtain the weight score of each feature subset;

[0028] Sorting the feature subsets in descending order of weight scores to obtain a feature subset sequence;

[0029] According to the feature subset sequence, sequential sampling is performed starting from the first feature subset to obtain a target number of feature subsets as target feature subsets.

[0030] In the above solution, the training module is also used to obtain the conversion relationship between the feature and the training sample of the linear model;

[0031] Based on the conversion relationship, the at least one feature is converted respectively to obtain a vector representation corresponding to each feature;

[0032] The vector representation is used as a training sample of the linear model.

[0033] In the above scheme, the training module is also used to use the prediction results as sample labels of the corresponding training samples to train the linear model and obtain model parameters when the linear model converges.

[0034] The embodiment of the present application provides a data processing method of a federated learning model, which is applied to a second participant device, including:

[0035] For each second feature used to train the federated learning model, generate an anonymous feature corresponding to each second feature;

[0036] sending the anonymous features corresponding to each of the second features to the first party device;

[0037] The anonymous feature is used for the first participant device to select at least one feature from at least one first feature used to train the federated learning model and the anonymous feature, to construct a training sample for the linear model, and based on the training sample and the prediction result of the federated learning model corresponding to the at least one feature, to train the linear model to obtain the model parameters when the linear model converges, and based on the model parameters, to determine the first contribution information of each of the first features corresponding to the prediction result, and the second contribution information of the anonymous feature corresponding to the prediction result.

[0038] The present application embodiment provides a data processing device for a federated learning model, including:

[0039] A generating module, configured to generate, for each second feature used to train the federated learning model, anonymous features corresponding to each second feature;

[0040] A sending module, used for sending the anonymous features corresponding to each of the second features to the first participant device;

[0041] The anonymous feature is used for the first participant device to select at least one feature from at least one first feature used to train the federated learning model and the anonymous feature, to construct a training sample for the linear model, and based on the training sample and the prediction result of the federated learning model corresponding to the at least one feature, to train the linear model to obtain the model parameters when the linear model converges, and based on the model parameters, to determine the first contribution information of each of the first features corresponding to the prediction result, and the second contribution information of the anonymous feature corresponding to the prediction result.

[0042] In the above solution, the generating module is further used to receive an anonymous feature acquisition request carrying a sample identifier sent by the first participant device;

[0043] Parsing the anonymous feature acquisition request to obtain the sample identifier;

[0044] Determine the second feature corresponding to the sample identifier and an anonymous feature corresponding to the second feature;

[0045] The anonymous characteristic is sent to the first party device.

[0046] In the above solution, the sending module is used to receive the ratio corresponding to the anonymous feature sent by the first participant device, and the ratio is used to represent the contribution ratio of the anonymous feature to the prediction result;

[0047] Based on the ratio, contribution information of the corresponding second feature is determined, and the contribution information is output.

[0048] An embodiment of the present application provides an electronic device, including:

[0049] A memory for storing executable instructions;

[0050] The processor is used to implement the data processing method of the federated learning model provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0051] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute and implement the data processing method of the federated learning model provided in the embodiment of the present application.

[0052] An embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the data processing method of the federated learning model provided in the embodiment of the present application.

[0053] The embodiments of the present application have the following beneficial effects:

[0054] Compared with the related technology of using the feature importance output by the federated tree model to interpret the machine learning model, in the embodiment of the present application, the first participant device constructs the training sample of the corresponding linear model through the first feature used locally for training the federated learning model and the anonymous feature received from the second participant device, trains the linear model, and obtains the model parameters when the linear model converges. In this way, the training process of the linear model can be concentrated locally, greatly reducing the communication overhead with the second participant device; then, according to the model parameters, the contribution information of the first feature provided by the first participant device and the anonymous feature provided by the second participant device is determined. In this way, the contribution information of each feature provided by each participant in the federated learning model can be accurately measured while ensuring data security, and at the same time, the calculation amount of the model can be greatly reduced, and the calculation efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a schematic diagram of the architecture of the data processing system of the federated learning model provided in the embodiment of the present application;

[0056] Figure 2A-2B is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0057] Figure 3 It is a flowchart of a data processing method of a federated learning model provided in an embodiment of the present application;

[0058] Figure 4 is a flow chart of the anonymous feature acquisition method provided in an embodiment of the present application;

[0059] Figure 5It is a flowchart of a feature subset acquisition method provided by the present application;

[0060] Figure 6 It is a schematic diagram of the target feature subset sampling process provided in an embodiment of the present application;

[0061] Figure 7 is another flowchart of the sampling process of the target feature subset provided in an embodiment of the present application;

[0062] Figure 8 It is a schematic diagram of the feature subset conversion process provided by an embodiment of the present application;

[0063] Fig. 9 is a schematic diagram of training samples of the linear model provided in the embodiment of the present application;

[0064] Fig.10 It is a schematic diagram of data processing of the federated learning model provided in an embodiment of the present application;

[0065] Fig.11 Schematic diagram of linear model feature contribution provided in the embodiment of the present application;

[0066] Fig.12 It is a flowchart of a data processing method of a federated learning model provided in an embodiment of the present application;

[0067] Fig.13 is a flow chart of a data processing method for a federated learning model provided in an embodiment of the present application;

[0068] Fig.14 is a schematic diagram of a subset matrix provided in an embodiment of the present application;

[0069] Fig.15 It is a schematic diagram of the subset sampling process provided in an embodiment of the present application;

[0070] Fig.16 It is a schematic diagram of a sample explanation flow provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0072] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0073] If similar descriptions of "first / second" appear in the application documents, the following instructions are added. In the following description, the terms "first\second\third" are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0075] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0076] 1) Shapley Additive exPlanation (SHAP): A model-independent interpretability analysis method based on collaborative game theory. Each prediction record has a corresponding shap value, and each feature also has a corresponding shap value. When the shap value is greater than 0, it means that the current feature in the current sample pushes the model prediction result in the positive direction, otherwise it pushes it in the negative direction.

[0077] 2) Median: Also known as the middle value, a special term in statistics. It is the number in the middle of a set of data arranged in order. It represents a value in a sample, population or probability distribution, which can divide the set of values ​​into two equal parts. For a finite set of numbers, you can sort all the observations and find the one in the middle as the median. If there is an even number of observations, the average of the two middle values ​​is usually taken as the median.

[0078] 3) Mode: refers to the value with a clear central trend point in the statistical distribution, representing the general level of the data. It is also the value that appears most frequently in a set of data. Sometimes there are several modes in a set of numbers, represented by M.

[0079] The embodiments of the present application provide a data processing method, device, electronic device, computer-readable storage medium and computer program product for a federated learning model, which can accurately measure the contribution information corresponding to the features provided by each participant in a single sample, and at the same time can significantly reduce the calculation amount of the model and improve the calculation efficiency.

[0080] Based on the above explanations of the nouns and terms involved in the embodiments of the present application, the data processing system of the federated learning model provided in the embodiments of the present application is first described. Figure 1 , Figure 1 It is an architectural diagram of the data processing system of the federated learning model provided in an embodiment of the present application. In the data processing system 100 of the federated learning model, the first participant device 400, the second participant device 410 (two second participant devices are shown as an example, respectively denoted as 410-1 and terminal 410-2, for distinction), the first participant device 400 and the second participant device 410 are connected to each other through a network 300 and are connected to a parameter aggregation device 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two, and a wireless link is used to realize data transmission.

[0081] In some embodiments, the first participant device 400 and the second participant device 410 are connected to each other via the network 300, and third-party devices (collaborators, servers, etc.) that may be involved in the federated learning model may be connected via the network 300.

[0082] In some embodiments, the first participant device 400 and the second participant device 410 can be a laptop, a tablet computer, a desktop computer, a smart phone, a dedicated messaging device, a portable gaming device, a smart speaker, a smart watch, etc., and can also be a client terminal of a federated learning participant, such as a participant device of each bank or financial institution that stores user feature data, but is not limited to this. The parameter aggregation device 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms, etc., to assist each participant device in performing federated learning to obtain a federated learning model. The network 300 can be a wide area network or a local area network, or a combination of the two. The first participant device 400 and the second participant device 410 can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiments of the present application.

[0083] The first participant device 400 is used to perform federated learning model training with the second participant device 410-1 and the second participant device 410-2 respectively to obtain a trained federated learning model.

[0084] The first participant device 400 is also used to obtain at least one first feature provided locally for training the federated learning model; receive at least one anonymous feature sent by the second participant device, wherein the anonymous feature corresponds to a second feature used to train the federated learning model; select at least one feature from the first feature and the anonymous feature, and obtain the prediction result of the federated learning model corresponding to each feature; based on at least one feature, construct a training sample of the linear model, and based on the training sample and the prediction result, train the linear model to obtain the model parameters when the linear model converges; based on the model parameters, determine the first contribution information of each first feature corresponding to the prediction result, and the second contribution information of each anonymous feature corresponding to the prediction result.

[0085] The second participant device 410 is used to send encrypted local features to the first participant device 400 for constructing training samples for the federated learning model. When receiving a notification message sent by the first participant device 400, the local features are filled with default values ​​or target values ​​to obtain model inputs corresponding to the local features.

[0086] The first participant device 400 is also used to send an anonymous feature acquisition request to the second participant device, so that the second participant device 410 responds to the anonymous feature acquisition request, determines the second feature corresponding to the sample identifier, and returns the anonymous feature corresponding to the second feature to the first participant device 400.

[0087] The second participant device 410 is used to generate anonymous features corresponding to each second feature used to train the federated learning model; and send the anonymous features corresponding to each second feature to the first participant device 400.

[0088] See also Figure 2A-2B , Figure 2A-2B is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. In practical applications, the electronic device 500 can be implemented as Figure 1 The first participant device 400 or the second participant device 410 in the embodiment of the present application is used to illustrate an electronic device for implementing the data processing method of the federal tree model. Figure 2A-2B The electronic device 500 shown includes: at least one processor 510, a memory 550, at least one network interface 520 and a user interface 530. The various components in the electronic device 500 are coupled together through a bus system 540. It can be understood that the bus system 540 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 540 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are marked as bus system 540 in FIG.

[0089] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0090] The user interface 530 includes one or more output devices 531 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0091] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 550 may optionally include one or more storage devices that are physically remote from the processor 510.

[0092] The memory 550 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 550 described in the embodiments of the present application is intended to include any suitable type of memory.

[0093] In some embodiments, the memory 550 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0094] Operating system 551, including system programs for processing various basic system services and performing hardware-related tasks, such as framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0095] A network communication module 552, for reaching other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB);

[0096] a presentation module 553 for enabling presentation of information via one or more output devices 531 (e.g., display screen, speaker, etc.) associated with the user interface 530 (e.g., a user interface for operating peripherals and displaying content and information);

[0097] The input processing module 554 is used to detect one or more user inputs or interactions from one of the one or more input devices 532 and translate the detected inputs or interactions.

[0098] In some embodiments, the data processing device of the federated tree model provided in the embodiments of the present application can be implemented in software. Figure 2A The electronic device provided in the embodiment of the present application is a schematic diagram of the structure of the first participant device, and the data processing device 555 of the federated tree model stored in the memory 550 can be software in the form of programs and plug-ins, including the following software modules: acquisition module 5551, receiving module 5552, selection module 5553, training module 5554 and determination module 5555. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0099] In some embodiments, Figure 2B As shown, Figure 2B The electronic device provided in the embodiment of the present application is a schematic diagram of the structure of the second participant device. The software modules stored in the data processing device 555 of the federated tree model of the memory 550 may include: a generation module 5556 and a sending module 5557. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0100] In other embodiments, the data processing device of the federal tree model provided in the embodiments of the present application can be implemented in hardware. As an example, the data processing device of the federal tree model provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the data processing method of the federal tree model provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs) or other electronic components.

[0101] The data processing method of the federated learning model provided in the embodiment of the present application will be described in combination with the exemplary application and implementation of the terminal provided in the embodiment of the present application. The method is applied to the first participant device, see Figure 3 , Figure 3 This is a flow chart of the data processing method of the federated learning model provided in the embodiment of the present application, which will be combined with Figure 3 The steps shown are explained.

[0102] In step 101, a first participant device obtains at least one first feature provided by the first participant device for training a federated learning model.

[0103] In actual implementation, in the scenario of the vertical federated tree model, at least two participants are usually involved, the first participant is the participant holding data and labels (also called the guest party or active party), and the second participant is the participant holding data (also called the host party or passive party). The method provided in the embodiment of the present application can be applied to a vertical federated learning model involving a guest party and at least one host party.

[0104] Exemplarily, taking a guest and at least one host co-training a federated learning model as an example, in a single training sample, the guest and the host each hold some features, wherein the guest provides G_1, G_2, G_3, ..., G_n, a total of n first features, n ≥ 1 and n is an integer; the host provides H_1, H_2, H_3, ..., H_m, a total of m second features, m ≥ 1 and m is an integer; if there are k hosts (k ≥ 1 and k is an integer), m_k can be used to represent the number of features included in the kth host. It can be understood that the feature set corresponding to a single training sample of a guest and a host contains a total of n first features and m second features, and the feature set can be expressed as {G_1, G_2, G_3, ..., G_n, H_1, H_2, H_3, ..., H_m}.

[0105] Among them, for the training samples of the federated learning model, at least two first features {G_1, G_2, G_3, ..., G_n} provided by the first participant device are public to the local (first participant device), and the first participant device can directly obtain the first features locally.

[0106] In step 102, at least one anonymous feature sent by a second participant device is received, wherein the anonymous feature corresponds to a second feature used to train a federated learning model.

[0107] In actual implementation, in order to protect the privacy and security of the data of each participant, for the m second features {H_1, H_2, H_3, ..., H_m} provided by the second participant device, the first participant device cannot directly obtain the information related to the second features. In order to obtain the contribution information of each second feature corresponding to the prediction result of the sample to be explained, the contribution information of each second feature can be determined by obtaining the anonymous feature corresponding to each second feature from the second participant device and determining the contribution information of the anonymous feature in the first participant device. Anonymous features (anonymousfeature) can be directly used locally to replace the second feature provided by the Host. At this time, the first participant device does not need to know the specific information of the second feature, but only needs to obtain the anonymous feature corresponding to the second feature. In other words, when the first participant device determines the contribution information of the second feature in the sample to be explained, it first needs to receive the anonymous feature corresponding to the second feature sent by the second participant device.

[0108] Continuing with the above example, the Guest party obtains the anonymous features sent by the Host party that correspond one-to-one to the m second features, and the feature set of the first participant device for a single training sample is changed to {G_1, G_2, G_3, …, G_n, a_1_i, a_2_i, a_3_i, …, a_m_i}, where a_m_i can represent the mth anonymous feature of the i-th Host party, and when i=1, it represents the first Host party.

[0109] In some embodiments, see Figure 4 , Figure 4 is a flow chart of the anonymous feature acquisition method provided in the embodiment of the present application. The first participant device can Figure 4 The steps shown are to obtain anonymous features, combined with Figure 4 The steps shown are explained.

[0110] Step 1021: The first participant device sends an anonymous feature acquisition request carrying a sample identifier to the second participant device.

[0111] In actual implementation, when the first participant device is interpreting a sample, when the feature in the sample is the second feature, an anonymous feature request can be sent to the second participant device, wherein the anonymous feature request carries a sample identifier, and the sample identifier is used to identify the sample to which the first feature belongs for training the federated learning model (the sample identifier corresponding to the sample to be interpreted). The purpose of the anonymous feature acquisition request is that the second participant device responds to the anonymous feature acquisition request to determine the second feature corresponding to the sample identifier and the anonymous feature corresponding to the second feature. It should be noted that the sample identifier carried in the anonymous feature request can be one or more. When the number of sample identifiers is multiple, the first participant device can batch obtain the anonymous features corresponding to the second features in multiple samples to be interpreted. In this way, the number of communications with the second participant device can be reduced, and at least two samples can be batch interpreted, thereby improving the efficiency of sample interpretation.

[0112] Step 1022: The first participant device receives the anonymous feature corresponding to the sample identifier returned by the second participant device.

[0113] In actual implementation, after the first participant device sends an anonymous feature request to the second participant device, the first participant device can receive the anonymous feature corresponding to the sample identifier returned by the second participant device. In addition, the first participant device can create an anonymous relationship record table locally to record the anonymous feature corresponding to the sample identifier. When the sample corresponding to the sample identifier is used again, it can be directly read from the local anonymous relationship record table, thereby reducing the number of communications with the second participant device and improving the data processing efficiency of the first participant device.

[0114] Exemplarily, the anonymous relationship record table can be stored locally in a file format, where the recorded information can be {"sample identification": ××××, "anonymous feature": ×××}, and can also include the device identification of the second party's device, such as {"sample identification": ××××, "anonymous feature": ×××, "device identification": host_x}.

[0115] It should be noted that there is no strict execution order between step 101 and step 102 .

[0116] In step 103, the first participant device selects at least one feature from the first feature and the anonymous feature, and obtains the prediction result of the federated learning model corresponding to each feature.

[0117] In actual implementation, after the first participant device executes step 102, for the sample to be interpreted, the features locally present are the first features provided locally and the anonymous features corresponding to the second features provided by the host party.

[0118] In some embodiments, the first participant device may select at least one feature from the first feature and the anonymous feature in the following manner: the first participant device constructs a feature set including at least one first feature and at least one anonymous feature; obtains a feature subset included in the feature set; samples from multiple feature subsets to obtain a target number of target feature subsets, and uses the feature included in the target feature subset as the at least one selected feature.

[0119] In actual implementation, the second feature originally provided by the second participant device in the feature set of the sample to be interpreted can be replaced by the anonymous features corresponding to the second feature, that is, the feature set of the local sample to be interpreted of the first participant device can be {G_1, G_2, G_3, ..., G_n, a_1_i, a_2_i, a_3_i, ..., a_m_i}, where a_m_i can represent the mth anonymous feature of the i-th host party. Based on this, the first participant device can obtain the contribution information of each feature in the feature set locally based on the first feature and the anonymous feature.

[0120] In actual implementation, the contribution information of each feature included in the sample feature set used as the machine learning model can be determined based on the Shapley value (Shapley). The implementation process can be: assuming that there is a machine learning model f, the input sample X, the sample X contains features {1, 2, 3, ..., t}, the feature can be replaced by the subscript i, there are t (t ≥ 1 and t is an integer) features in total, N is a subset, S is a feature subset that does not contain i, f x It is an estimation function that returns the average value of the feature subset S output by the model f. The calculation formula is as follows:

[0121]

[0122] As an example, suppose there is a sample x = {age = 20, height = 170, income = 100}, and the machine learning model f obtains the prediction result of f for sample x as 1.2.

[0123] 1) Enumerate all feature subsets included in the feature set of sample x: {age = 20, height = 170, income = 100}, {age = 20, height = 170}, {age = 20, income = 100}, {height = 170, income = 100}, {income = 100}, {age = 20}, {height = 170}, {}.

[0124] 2) Traverse each feature in sample x, add the feature value corresponding to each feature to the feature subset that does not contain itself, calculate the marginal contribution of each feature, and use the preset valuation function f_x to calculate the prediction result (also called estimated value) corresponding to each feature subset. Assume that through f_x, the prediction results of each feature subset are as follows:

[0125] f_x({age=20, height=170, income=100})=1.2;

[0126] f_x({age=20, height=170})=0.9;

[0127] f_x({age=20, income=100})=0.8;

[0128] f_x({height=170, income=100})=0.7;

[0129] f_x({income=100})=0;

[0130] f_x({age=20})=0;

[0131] f_x({height=170})=0;

[0132] f_x({})=0;

[0133] It should be noted that when the feature subset is the full set of features in sample x, the prediction result calculated using the valuation function f_x is consistent with the prediction result using the machine learning model f.

[0134] Continuing with the above example, in order to calculate the contribution of the feature "{age = 20}" to the prediction result "1.2" of sample x, "{age = 20}" can be added to the feature subset that does not contain itself, and the marginal contribution value of "{age = 20}" can be calculated according to the above formula (1). The process is as follows:

[0135] ① Add "{age = 20}" to the feature subset {height = 170, income = 100}, and determine the marginal contribution value as: 2 / 6*{1.2-0.7}=1 / 6;

[0136] ② Add "{age = 20}" to the feature subset {income = 100} and determine the marginal contribution value as: 1 / 6*{0.8-0}=0.8 / 6;

[0137] ③ Add "{age = 20}" to the feature subset {age = 20}, and determine the marginal contribution value as: 1 / 6*{0.9-0}=0.9 / 6;

[0138] ④ Add "{age=20}" to the empty set and determine the marginal contribution value to be: 0;

[0139] Adding the marginal contribution values ​​obtained from ① to ④ above, the total contribution is 2.7 / 6=0.45. Finally, it is obtained that "{age=20}" contributes "+0.45" to the prediction result "1.2". Similarly, it can be obtained that "{income=100}" contributes "+0.35" to the prediction result "1.2", and "{height=170}" contributes "+0.4" to the prediction result "1.2". Among them, the symbol "+" indicates that "{age=20}", "{income=100}", and "{height=170}" have a positive impact on the prediction result "1.2", and if the symbol is "-", it indicates that the feature value corresponding to the feature has a negative impact on the target prediction result.

[0140] From the above description, it can be determined that when the first participant device determines the contribution information of each feature included in the feature set including the first feature and the anonymous feature to the prediction result of the sample based on the Shapley value. First, it is necessary to determine each feature subset included in the feature set including the first feature and the anonymous feature. Next, the method of obtaining the feature subset is explained. The feature subset is obtained by combining each feature in the feature set. The calculation formula for the number of feature subsets N is as follows:

[0141]

[0142] Among them, n+m is the number of features, Indicates the number of feature subsets containing 0 features (that is, the number of empty sets is 1), represents the number of feature subsets containing 1 feature, It represents the number of feature subsets containing n+m features (i.e., the number of the full set is 1). In practical applications, the empty set can be ignored.

[0143] In actual implementation, in the federated learning model, it is assumed that a guest and a host participate in the calculation. For each sample D containing n+m features, n is the number of first features provided by the guest, and m is the number of second features provided by the host. For sample D, in order to ensure the contribution of obtaining relevant features of the host, while ensuring the data security between the participating parties, the m second features provided by the host are replaced by m corresponding anonymous features. It should be noted that there is a one-to-one correspondence between anonymous features and second features. For a sample to be explained, there are several anonymous features corresponding to several second features, and each anonymous feature has a unique identifier. The anonymous feature can be recorded as a_m_i, where i is used to represent the index of the host, and m is used to identify the mth anonymous feature provided by the kth host. In actual applications, there can be multiple hosts. The feature set {G_1, G_2, G_3, ..., G_n, H_1, H_2, H_3, ..., H_m} corresponding to the sample to be explained is changed to the feature set D'{G_1, G_2, G_3, ..., G_n, a_1_1, a_2_1, a_3_1, ..., a_m_1} corresponding to the sample to be explained. For the n+m features included in the sample D', the n+1 features are combined and calculated according to the above formula (2) to obtain the 2 corresponding to the sample D' n+m feature subset.

[0144] For example, a training sample S {age: 28, education: undergraduate, income: 10,000, work: IT} used as a federated learning model, the prediction result obtained by inputting the sample into the federated learning model indicates that the customer is a defaulting customer, wherein the feature {age: 28, education: undergraduate} is the first feature provided by the Guest, and the feature {income: 10,000, work: IT} is the second feature provided by the Host. In the federated learning model, the Guest cannot obtain the specific information of each feature of the Host. In practical applications, when interpreting this sample, in order to ensure the security of the Host's data, the features {income: 10,000, work: IT} can be replaced by corresponding anonymous features (a_1_1, a_2_1), then the feature set corresponding to this training sample contains 2 features {age: 28, education: undergraduate} provided by the Guest, and two anonymous features, for a total of 4 features. The feature subset corresponding to this training sample contains 0 to 4 features, for a total of 16 (2 4 ) feature subsets.

[0145] It should be noted that since the feature subset with incomplete features cannot directly obtain the prediction results through the machine learning model (the federated learning model f in the embodiment of the present application), it is necessary to fill in the missing features and construct the model input (or training sample) corresponding to the federated learning model f. In some embodiments, the missing features can be filled in the following ways: determining the default value corresponding to each missing feature; assigning feature values ​​to the missing features based on the default value.

[0146] In actual implementation, the default value of the missing feature can be set directly according to the actual situation, and a fixed value can be used as the default value of the missing feature.

[0147] Continuing with the above example, for the feature subset {age: 30, a_1_1, a_2_1} of the training sample S'{age: 28, education: bachelor's degree, a_1_1, a_2_1}, the missing feature is {education level}. The default value of the feature "education level" can be pre-set to "bachelor's degree", and {education level: bachelor's degree} can be filled in the feature subset.

[0148] In some embodiments, the default value corresponding to the missing feature can also be determined in the following manner: obtain a set of training samples used as a federated learning model; perform the following processing for each feature subset: obtain the feature value corresponding to the missing feature of the feature subset in each training sample of the training sample set; determine the target value corresponding to the missing feature based on the feature value corresponding to the missing feature in each training sample; and fill in the target value corresponding to the corresponding missing feature in the target feature subset.

[0149] For example, the features corresponding to the training sample are {income = 100, age = 20, height = 170}, and the feature subset is {age = 20, height = 170}. Obviously, the missing feature is income, and the machine learning model cannot predict the sample corresponding to this feature subset. It can be filled with the mean / mode / median of "income" (which can be calculated from the training data), and the predicted value of the filled sample is used as an estimate of the average output of the model. From the training sample set, the average value of the "income" feature is calculated to be 50, the average value of the "height" feature is calculated to be 150, and the average value of the "age" feature is calculated to be 15. Then {income = 50} can be used to fill the feature subset {age = 20, height = 170} to obtain the corresponding filled sample {age = 20, height = 170, income = 50}.

[0150] Secondly, the first party device also needs to determine the valuation function used to calculate the prediction results of each feature subset. Since each feature is added to the feature subset that does not include itself to calculate the marginal contribution, and then the marginal contribution is summed to obtain the feature contribution information, strictly following the above Shapely calculation process, all subsets are enumerated. The number of subsets is very large, and the amount of calculation and communication is huge. Therefore, in actual implementation, the target number of feature subsets can be sampled from all feature subsets of the corresponding feature set, and based on the feature subsets and the prediction results of the feature subsets by the valuation function, a linear model is trained to calculate the contribution information of each feature, which can greatly reduce the computational overhead.

[0151] In some embodiments, see Figure 5 , Figure 5 This is a flow chart of the feature subset acquisition method provided by the present application, combined with Figure 5 The steps shown are explained.

[0152] Step 201: The first participant device obtains the weight score of each feature subset.

[0153] In actual implementation, the feature set in each training sample used as the federated learning model corresponds to multiple feature subsets with different subset sizes, where the subset size refers to the number of features contained in the feature subset. Scoring subsets of different sizes can preliminarily estimate the importance of each feature.

[0154] The method of scoring the feature subsets is described. In some embodiments, the weight score of each feature subset may be determined by the following formula:

[0155] W(m)=(M-1) / (m*(Mm)) (3)

[0156] Among them, the weight score of the subset size m is represented by W(m), M is the number of features in a training sample used as a federated learning model, m is a positive integer greater than 0, and M>m.

[0157] Traverse the feature subsets of different sizes corresponding to the feature set, and score each feature subset according to the above formula (3). Finally, all subsets of sizes from 1 to M-1 are scored, and a weight score set is obtained, which can be regarded as a scoring vector.

[0158] In some embodiments, the number of elements in the obtained scoring vector may be M-1, that is, the scoring is performed according to the size of the subset. Ignoring the empty set and the full set, the subset size corresponding to a feature set containing M features is from 1 to M-1, and accordingly, the number of elements in the obtained scoring vector is also M-1, that is, weight = [w_1, ..., w_(M-1)], where w_1 represents the weight score of the subset with a subset size of 1, and w_(M-1) represents the weight score of the subset with a subset size of M-1.

[0159] In some embodiments, ignoring the empty set and the full set, the number of elements in the obtained scoring vector may be all feature subsets corresponding to the positive set containing M features, where the number of feature subsets corresponding to each subset size is represented using the following combination function.

[0160] C(r,t)=t! / r!(tr)!(4)

[0161] Where r is an integer, t is a positive integer greater than or equal to 1, and t>r. It can be interpreted as the number of feature subsets C(r,t) with a subset size of r corresponding to a feature set containing t features, for example, C(2,5)=10.

[0162] For example, taking M=5 as an example, the number of subsets with one element in the subset is C(1,5) is 5, the number of subsets with two elements in the subset is C(2,5), the number of subsets with three elements in the subset is C(3,5), and the number of subsets with four elements in the subset is C(4,5). Ignoring the empty set and the full set, the corresponding total number of feature subsets is: Total=C(1,5)+C(2,5)+C(3,5)+C(4,5)=5+10+10+5=30. The above formula (2) is used to score the feature subsets of all subset sizes from 1 to 4, that is:

[0163] The weight of a subset of size 1 can be calculated as: (5-1) / (1*4)=1;

[0164] The weight of a subset of size 2 can be calculated as: (5-1) / (2*3)=4 / 6;

[0165] The weight of a subset of size 3 can be calculated as: (5-1) / (3*2)=4 / 6;

[0166] The weight of a subset of size 4 can be calculated as: (5-1) / (4*1)=1.

[0167] Finally, we get a scoring vector weight. Weight can be expressed in two forms: one is weight = [1, 4 / 6, 4 / 6, 1]; the other is:

[0168]

[0169] It should be noted that, from the property of W(m) in formula (4), it can be seen that the weights of larger and smaller subsets are larger, and the weights of complementary sets are equal.

[0170] Continuing with the above example, taking M=5 as an example, the weight score value of W(1)=W(4) is 1, and the weight score value of W(2)=W(3) is 4 / 6.

[0171] Step 202, sorting each feature subset in descending order of weight score to obtain a feature subset sequence.

[0172] In actual implementation, after obtaining the weight score corresponding to each feature subset, the obtained scoring vectors can be directly reordered to obtain a weight score sequence in which the weight scores are sorted from large to small.

[0173] Continuing with the above example, taking M=5 as an example, the corresponding weight=[1,4 / 6,4 / 6,1]. After reordering, we get weight'=[1,1,4 / 6,4 / 6]. It can be determined that the weight scores of W(1)=W(4) are equal and higher, and the weight scores of W(2)=W(3) are equal and lower.

[0174] Step 203 , sampling is performed sequentially starting from the first feature subset according to the feature subset sequence to obtain a target number of feature subsets as target feature subsets.

[0175] In actual implementation, according to the feature subset sequence, sequential sampling is performed starting from the first feature subset to obtain a target number of target feature subsets. The first number may be a preset sampling number, and the value of the first number is less than the total number of feature subsets, which may be represented by max_subset.

[0176] Continuing with the above example, taking M=5 as an example, if the preset sampling number is 10, the feature subsets with a subset size of 1, the number of which is C(1,5)=5, and the feature subsets with a subset size of 4, the number of which is C(4,5)=5, can be directly used as target feature subsets, totaling 10, which is consistent with the preset sampling number, and the sampling is completed.

[0177] In some embodiments, the target feature subset can also be sampled by the size of the scale factor. Figure 6 , Figure 6 Schematic diagram of the target feature subset sampling process provided in the embodiment of the present application. Figure 6 The steps shown are explained.

[0178] Step 301: The first participant device performs regularization processing on the weight scores of each feature subset to obtain a proportional coefficient corresponding to each weight score.

[0179] It should be noted that the proportional coefficient is actually a regularization (normalization) operation performed on the weight scores corresponding to each feature subset obtained according to formula (3) above, to obtain a proportional coefficient with a value range between 0 and 1. The larger the proportional coefficient, the higher the corresponding weight score.

[0180] In some embodiments, according to the score vector weight obtained by the above calculation, the proportion of the weight of each feature subset to the total is calculated, and a proportion coefficient is obtained to form a proportion vector, which is denoted as p. The calculation formula is as follows:

[0181] p=weight / (w_1+……+w_(M-1))=[p1,p2,……,p_(M-1)] (5)

[0182] For example, taking M=5 as an example, through the above formula (3), when the weight fraction weight=[1,4 / 6,4 / 6,1] corresponding to each feature size is obtained, then according to formula (5), each weight size in weight is regularized (normalized) to obtain the p value corresponding to each feature subset size:

[0183] The value corresponding to the weight of the feature subset of size 1 is: p_1 = 1 / (1+4 / 6+4 / 6+1) = 0.3;

[0184] The value corresponding to the weight of the feature subset of size 2 is: p_2 = (4 / 6) / (1+4 / 6+4 / 6+1) = 0.2;

[0185] The value corresponding to the weight of the feature subset of size 3 is: p_3 = (4 / 6) / (1+4 / 6+4 / 6+1) = 0.2;

[0186] The value corresponding to the weight of the feature subset of size 4 is: p_4 = 1 / (1+4 / 6+4 / 6+1) = 0.3;

[0187] Finally, we get the scale vector p = [0.3, 0.2, 0.2, 0.3]. In addition, when the weight scores corresponding to the size of each feature are obtained, the following expression is obtained:

[0188]

[0189] The obtained proportional coefficient set p is expressed as follows:

[0190]

[0191] Step 302: Based on the proportional coefficient corresponding to each weight score, multiple target feature subsets are sampled from the multiple feature subsets.

[0192] After obtaining the weight scores corresponding to each feature subset and performing regularization processing to obtain the corresponding proportional coefficients, in some embodiments, the target feature subset can be sampled in the following manner: to sample the target feature subset by proportional coefficients, it is necessary to first sort the proportional coefficients corresponding to each weight score according to the size of the proportional coefficients to obtain a proportional coefficient sequence; then, according to the sorting of the proportional coefficients in the proportional coefficient sequence, perform corresponding processing on each proportional coefficient in turn. Figure 7 , Figure 7 is another flow chart of the sampling process of the target feature subset provided in the embodiment of the present application, based on Figure 6 , step 302 can be done by Figure 7 The steps shown are implemented.

[0193] In step 3021, the first participant device obtains the current sampling number, determines the product of the proportional coefficient and the current sampling number, and uses the product as the current capacity value.

[0194] In actual implementation, the current sampling number can be represented by i_subset, where i represents the index of the current operation's proportional coefficient in the proportional coefficient sequence, and can also be used to represent the current sampling round. If i starts at 1, 1_subset means that the current operation's proportional coefficient is the first proportional coefficient in the proportional coefficient sequence, or the first round of sampling. It should be noted that when the proportional coefficient currently being processed is the first proportional coefficient in the proportional coefficient sequence, the current sampling number is the preset sampling threshold, represented by max_subset, that is, 1_subset = max_subset. In order to obtain a reasonable subset of target features, the maximum number of samples for the current sampling round is calculated in each sampling round.

[0195] In some embodiments, the maximum number of target feature subsets that can be sampled in the current sampling round is determined based on the product of the current proportional coefficient p_current and the current sampling number i_subset, which can also be called the capacity of the current round and is represented by capacity, that is, capacity = i_subset*p_current.

[0196] For example, taking M=5, the sorted p=[p_1, p_(M-1), p_2, p_(M-2)] is obtained, that is, p=[p_1, p_4, p_2, p_3]=[0.3, 0.3, 0.2, 0.2]. When the first round of sampling is performed, p_curren=p_1=0.3, the corresponding subset size is p_size=1, i_subset=max_subset (assuming it is 20), and a capacity number is obtained: capacity=20*0.3=6.

[0197] Step 3022, obtaining the number of feature subsets corresponding to the proportional coefficient.

[0198] Continuing with the above example, when the first round of sampling is performed, the number of feature subsets corresponding to the subset size p_size=1 is C(1,5)=5. If the conditions for the second sampling are met, the number of feature subsets corresponding to the subset size p_size=4 is C(4,5)=5.

[0199] Step 3023, when the current capacity value is greater than the quantity, the feature subset corresponding to the proportional coefficient is used as the target feature subset.

[0200] Continuing with the above example, in the first round of sampling, capacity = 6, that is, the number of feature subsets that can be accommodated in the current round is 6. At this time, the number of feature subsets corresponding to the subset size p_size = 1 is C(1,5) = 5, 5 is smaller than 6, therefore, all feature subsets of p_size = 1 are sampled as target feature subsets. At this time, the number of target feature subsets that need to be sampled is updated to max_subset-5 = 15, that is, when the second round of sampling is performed, i_subset = 2_subset = 15.

[0201] Step 3024, when the current capacity value is less than the number, a feature subset with the same number as the current sampling number is randomly selected from the unselected feature subsets as the target feature subset.

[0202] Continuing with the above example, taking M=5 as an example, after the first round of sampling, the number of sampled target feature subsets is 5, and the second round of sampling needs to be continued, that is, executing steps 201 to 203. It should be noted that at this time, i_subset (i.e., 2_subset) is no longer max_subset(20), but is updated to 20-5=15, i.e., 2_subset=15, then the capacity obtained in step 202 is 15*0.3=4.5, and 4.5 is rounded to 4, i.e., the number of feature subsets that can be accommodated in the second round of sampling is 4, and at this time, the number of feature subsets corresponding to the subset size p_size=4 is C(4,5)=5, 5 is larger than 4, therefore, in the second round of sampling, all feature subsets of p_size=4 cannot be sampled as feature subsets. In some embodiments, in order to ensure the balance of the target feature subsets, the current traversal can be terminated directly, and random sampling is performed from all remaining feature subsets to obtain the remaining target feature subsets.

[0203] Continuing with the above example, all feature subsets of p_size=4 cannot be sampled as feature subsets. At this time, for the remaining 15 target feature subsets that need to be sampled, random sampling can be performed from the remaining feature subsets (30-5=25), and finally max_subset number of target feature subsets are obtained.

[0204] For each proportional coefficient in the proportional coefficient sequence, the proportional coefficients in p are traversed in turn, and the above steps 3021 to 3024 are executed in a loop until a target feature subset with a target sampling quantity is obtained.

[0205] In step 104, a training sample of the linear model is constructed based on at least one feature, and the linear model is trained based on the training sample and the prediction result to obtain model parameters when the linear model converges.

[0206] In some embodiments, the first participant device can construct a training sample of the linear model in the following manner: the first participant device obtains the conversion relationship between the feature and the training sample of the linear model; based on the conversion relationship, converts at least one feature separately to obtain a vector representation corresponding to each feature; and uses the vector representation as a training sample of the linear model.

[0207] In actual implementation, the conversion relationship between the feature subset corresponding to at least one feature and the training sample can be that each feature in the feature subset is represented by 0 and 1 to obtain the training sample of the linear model, that is, each feature in the training sample for the linear regression model h is represented by 0 and 1, where 1 represents the feature actually existing in the feature subset, and 0 represents the missing feature in the feature subset.

[0208] In some embodiments, the specific method of converting the features in the feature subset corresponding to at least one feature can be found in Figure 8 , Figure 8 This is a schematic diagram of the feature subset conversion process provided by the embodiment of the present application, combined with Figure 8 The steps shown are explained.

[0209] Step 401: The first participant device compares the target feature subset with the feature set, and obtains features that distinguish the target feature subset from the feature set as missing features.

[0210] In actual implementation, the missing features corresponding to the feature subset are determined by comparing the feature subset with the features in the training samples used as the federated learning model f.

[0211] Step 402 , respectively assigning feature values ​​to the missing features, and filling the missing features of the target feature subset based on the assignment results, to obtain target training samples of the linear model.

[0212] In actual implementation, after determining the missing features in the feature subset, 0 is used to represent the missing features, and 1 is used to represent the normal features in the feature subset.

[0213] In some embodiments, see Fig. 9 , Fig. 9It is a schematic diagram of the training samples of the linear model provided in the embodiment of the present application. The max_subset number of training samples can form a matrix of max_subset rows, each row represents a training sample used as a linear model, and the features of the training samples are represented by 0 or 1. Taking the first row as an example, assuming that there is a federated learning model in which a Guest and a Host participate, the Guest provides p-1 features {X1, X2, ..., Xp-1}, and the Host provides an anonymous feature Xp corresponding to the second feature. In the target feature subset sampled from these p features, X1, X2, ..., Xp, etc. are normal features represented by 1, and X1, X2, ..., Xp are missing features of the target feature subset represented by 0.

[0214] In some embodiments, the first participant device can train the linear model in the following manner: use a feature subset corresponding to at least one feature, corresponding to the prediction result of the federated learning model, as the sample label of the corresponding training sample, train the linear model, and obtain the model parameters when the linear model converges.

[0215] In actual implementation, the first participant device constructs a training sample of the linear model h (model input of the linear model) based on at least one feature selected in step 103, and uses each feature subset corresponding to at least one feature and the prediction result of the federated learning model f as the label data of the linear model h (model output of the linear model), trains the linear model, and obtains the model parameters when the linear model converges. Among them, the training method for the linear model can refer to the model training method in the relevant technology, and the embodiment of the present application does not limit the training method of the linear model.

[0216] In some embodiments, the linear model h can be represented by the following function:

[0217]

[0218] The guest provides n first features and anonymous features corresponding to the m second features of the host.

[0219] It should be noted that since the eigenvalues ​​of the training samples corresponding to the linear model are represented by 0 or 1, when the linear model reaches the convergence condition, that is, after the max_subset number of training samples are involved in the model training, the linear model converges and the model parameters are as follows: {w1,w2,……,w n ,w h_1 ,……,w h_k}.

[0220] In step 105, based on the model parameters, first contribution information of each first feature corresponding to the prediction result and second contribution information of each anonymous feature corresponding to the prediction result are determined.

[0221] In some embodiments, the obtained model parameters include contribution information of each feature in the feature set corresponding to the target prediction result. For example, a linear mapping relationship corresponding to a linear model is obtained, in which the prediction result is a dependent variable, each feature in the feature set is an independent variable, and the model parameters are coefficients of the independent variables; based on the model parameters and the linear mapping relationship, the contribution information of each feature in the feature set corresponding to the target prediction result is determined.

[0222] For example, for a 22-dimensional linear model,

[0223] h(x)=w1x1+w2x2+……+w 20 x 20 +w h_1 x h_1 +w h_2 x h_2

[0224] When the value of x is all 1, that is:

[0225] h(x)=w1×1+w2×1+……+w 20 ×1+w h_1 ×1+w h_2 ×1

[0226] The obtained linear model parameters {w1,w2,……,w 20 ,w h_1 ,w h_2}, that is, there are 22 weights, the first 20 weights {w1,w2,……,w 20}, can be directly approximated as the contribution information of the eigenvalues ​​corresponding to the first features used to train the federated learning model. The last two weights {w h_1 ,w h_2} can be directly approximated as the contribution information of the two anonymous features provided by the Host.

[0227] In some embodiments, see Fig.10 , Fig.10 This is a schematic diagram of federated learning model data processing provided in an embodiment of the present application. After obtaining the contribution information of each first feature and the contribution information of each anonymous feature, the first participant device can also execute steps 106 and 107.

[0228] Step 106: The first participant device sums the first contribution information of the prediction results corresponding to the first features and the second contribution information of the prediction results corresponding to the anonymous features to obtain the aggregated contribution information.

[0229] Step 107, respectively obtain the ratio of each second contribution information to the summary contribution information, and send each ratio to the second participant device, so that the second participant device determines the contribution information of the second feature corresponding to each anonymous feature based on the ratio.

[0230] In actual implementation, since the prediction result of the sample is sensitive information, it is safer for the second participant device (Host) to obtain a calculated ratio value. Based on this, the Guest adds the contribution information of all features (including each first feature and each anonymous feature) to obtain the summary contribution information, and then divides the contribution information of each anonymous feature of the Host by this summary contribution information to obtain a ratio (ratio value), and sends the ratio value of each anonymous feature to the corresponding Host, which maps it to the second feature used as the federated learning model. In this way, the Host can obtain the contribution information of each second feature.

[0231] For example, see Fig.11 , Fig.11 : This is a schematic diagram of the contribution of linear model features provided in an embodiment of the present application, which is used as a training sample for a federated learning model. The features provided by the Guest are {age=20, height=170, income=100}, and the features provided by the Host are {income=100} (here, the income feature is unknown to the Guest and can be represented by a_1_1, i.e., the first anonymous feature of the Host. The Host can determine that a_1_1 corresponds to its own second feature "income=100"). The prediction result of the corresponding federated learning model f is 1.2. The linear model is trained, h=0.45+0.4+0.35=1.2. It can be determined that the feature "age=20" contributes 0.45 to the prediction result 1.2; the feature "height=170" contributes 0.4 to the prediction result 1.2; and the anonymous feature a_1_1 contributes 0.35 to the prediction result 1.2. For data security considerations, the Guest obtains the contribution ratio corresponding to the anonymous feature a_1_1, 0.35 / 1.2≈0.29. The Guest sends "0.29" to the Host. Based on the association between the anonymous feature and the second feature stored locally, the Host can determine that the contribution information of the second feature "income = 100" corresponding to the anonymous feature a_1_1 relative to the predicted result 1.2 is 0.29.

[0232] In the embodiment of the present application, the first participant device constructs the training sample of the corresponding linear model through the first feature used locally for training the federated learning model and the anonymous feature received from the second participant device, trains the linear model, and obtains the model parameters when the linear model converges. In this way, the training process of the linear model can be concentrated locally, greatly reducing the communication overhead with the second participant device; then, according to the model parameters, the contribution information of the first feature provided by the first participant device and the anonymous feature provided by the second participant device is determined. In this way, the contribution information of each feature provided by each participant in the federated learning model can be accurately measured while ensuring data security, and at the same time, the calculation amount of the model can be greatly reduced, and the calculation efficiency can be improved.

[0233] Next, the data processing method of the federated tree model provided in the embodiment of the present application will be described in combination with the exemplary application and implementation of the second participant device provided in the embodiment of the present application. Fig.12 , Fig.12 This is a flow chart of the data processing method of the federated learning model provided in the embodiment of the present application, which will be combined with Fig.12 The steps shown are explained.

[0234] In step 501, the second participant device generates anonymous features corresponding to each second feature for training the federated learning model.

[0235] In some embodiments, the second participant device can generate an anonymous feature corresponding to the second feature in the following manner: the second participant device performs hash processing on each second feature used to train the federated tree model, obtains a hash value corresponding to each second feature, and uses the hash value as the anonymous feature corresponding to the second feature.

[0236] In actual implementation, the second participant device can set the anonymous feature of the second feature provided locally in each sample to be interpreted in a format agreed upon by the participants participating in the training of the federated tree model; the second participant can also generate the anonymous feature corresponding to the second feature by setting a non-repeating random number, which can be recorded as a_j_0, a_j_1, a_j_m, where a_j_m represents the mth feature of the jth host. The second participant can also calculate the hash value corresponding to the second feature through a common hash algorithm, and then record the obtained hash value as the anonymous feature corresponding to the second feature.

[0237] In step 502, the second party device sends anonymous features corresponding to each second feature to the first party device.

[0238] It should be noted that the anonymous feature is used for the first participant device to select at least one feature from at least one first feature used to train the federated learning model and the anonymous feature, to construct a training sample for the linear model, and to train the linear model based on the training sample and the prediction result of the federated learning model corresponding to at least one feature, to obtain the model parameters when the linear model converges, and to determine the first contribution information of the prediction result corresponding to each first feature and the second contribution information of the prediction result corresponding to the anonymous feature based on the model parameters.

[0239] In some embodiments, the second party device may send the anonymous feature in the following manner: the second party device receives an anonymous feature acquisition request carrying a sample identifier sent by the first party device; analyzes the anonymous feature acquisition request to obtain the sample identifier; determines the second feature corresponding to the sample identifier and the anonymous feature corresponding to the second feature; and sends the anonymous feature to the first party device.

[0240] In some embodiments, the second participating mode device may determine the second feature corresponding to the sample identifier and the anonymous feature corresponding to the second feature in the following manner: the first participating party device searches the local anonymous relationship record table according to the sample identifier to obtain the second feature corresponding to the sample identifier and the anonymous feature corresponding to the second feature. It should be noted that the anonymous relationship record table is used to record the sample identifier of the training sample used to train the federated learning model, the second feature corresponding to the sample identifier, and the anonymous feature corresponding to the second feature.

[0241] In actual implementation, the second participant device can store the second feature in each sample to be interpreted, the anonymous feature corresponding to the second feature, and the sample identifier of the sample to be interpreted in a local anonymous relationship record table. The anonymous relationship record table can be a two-dimensional relationship table stored in a database or a file stored locally.

[0242] In actual implementation, after sending the second feature corresponding to the sample identifier to the first participant device, the second participant device may also perform steps 403-404 to determine the contribution information of the local second feature.

[0243] Step 503: The second participant device receives the ratio corresponding to the anonymous feature sent by the first participant device, where the ratio is used to represent the contribution ratio of the anonymous feature to the prediction result.

[0244] Step 504: Determine contribution information of the corresponding second feature based on the ratio, and output the contribution information.

[0245] In actual implementation, after receiving the contribution ratio of the anonymous feature sent by the first participant device, the second participant device determines the second feature corresponding to the anonymous feature based on the association between the sample identifier, the anonymous feature and the second feature stored locally by the second participant, and uses the contribution ratio of the anonymous feature as the contribution ratio of the corresponding second feature to the prediction result of the federated learning model corresponding to the sample to be explained. The contribution ratio is output for use by other related businesses.

[0246] Following the above example, see Fig.11 , the Host receives the contribution ratio of 0.35 / 1.2≈0.29 corresponding to the anonymous feature a_1_1 sent by the Guest. Based on the association between the locally stored anonymous feature and the second feature, the Host can determine that the contribution information of the second feature "income=100" corresponding to the anonymous feature a_1_1 relative to the prediction result 1.2 is 0.29, and output it.

[0247] In the embodiment of the present application, the second participant creates an anonymous relationship record table locally, saves the second feature corresponding to the sample identifier and the anonymous feature corresponding to the second feature, and can directly search the anonymous feature corresponding to the sample identifier from the anonymous relationship record table when receiving an anonymous feature acquisition request, thereby improving query efficiency and reducing the number of communications with other participant devices.

[0248] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0249] The feature importance solution provided by the relevant vertical federation tree model can meet some requirements. Taking a vertical federation tree scenario involving two parties (Guest and Host) as an example, the feature importance solution provided by the vertical federation tree model is briefly described as follows:

[0250] (1) The guest initializes a table locally, which contains all the local features of the guest and the anonymous feature numbers sent by the host. The count value of each feature is 0.

[0251] (2) Start building a decision tree. For each split feature used by each decision tree node, add 1 to the corresponding count value in the table, or add the split gain value;

[0252] (3) After the decision tree is built, the feature importance table is output. The feature importance table can be used to explain the model: features with high count values ​​represent features that play a high role in the modeling process.

[0253] However, the feature importance scheme provided by the vertical federated tree model can meet the above requirement (1), but cannot meet the above requirements (2) and (3).

[0254] First, for the above requirement (2), users cannot use feature importance to specifically explain a single sample. For example, there is a sample S {age: 28, marital status: married, income: 10,000, education: undergraduate, work: IT, place of origin: ... and other features}, which is a defaulting customer. The vertical federated learning model is used to predict this sample and the score obtained is 0.1. Business users want to know how much each specific feature value in sample S (such as age = 28) contributes to the final prediction score of 0.1, and whether the feature has a positive or negative impact on the prediction score. In this way, the model can be combined with real-life experience to obtain some business inspiration.

[0255] It is impossible to judge the impact of a specific feature value on the model output and the positive or negative impact based solely on the obtained feature importance. This is because feature importance only reflects the usage of a global feature and cannot be used to analyze a specific sample.

[0256] For requirement (3) above, although feature importance can tell how many times the partner's feature is used, the positive and negative impact of the partner's feature on a single sample is unknown. If the partner's feature can provide a large impact on many samples, then it can be used to measure the value of the partner's feature. In addition, only the tree model can provide such a function, and other vertical federation models do not support similar feature importance analysis.

[0257] In order to solve the problems existing in the above-mentioned related technologies, the embodiment of the present application provides a data processing method for a federated learning model, that is, combining the principle of Shapely value analysis and optimizing the federated learning scenario, while reducing the amount of calculation and transmission, providing an explanation function for the contribution of each feature of each participant to the prediction result of the sample, and summarizing the explanations of multiple samples, it can provide a global feature importance analysis, and the results of feature importance can be shared with each participant. In this way, the contribution information of each feature of all participants can be obtained, and the feedback results of the host party can be provided at the same time, and the explanation of multi-classification machine learning models can also be supported.

[0258] First of all, before explaining the specific implementation process of the data processing method of the federated learning model, it is necessary to explain the relevant knowledge of SHAP.

[0259] SHAP value is a machine learning model interpretation scheme based on Shapley value (Shapely). Shapely value is a method of measuring the contribution of each participant (such as multiple companies) in game theory. It calculates the contribution of each party by considering the contribution of each party to the subset of participants that does not include itself:

[0260]

[0261] Among them, {x1,…,x p} is the set of all input features, p is the number of all input features, {x1,…,x p}\{x j} does not include x j is the possible set of all input features, val is a valuation function, val(S) is the prediction of feature subset S, |S| is the number of features in subset S, φ j (val) refers to the final calculated feature x j The Shapley value (SHAP value) of .

[0262] For example, suppose there are companies 1, 2, and 3, which work together and make a total profit of 1.2 million. The Shapely value is calculated for each company.

[0263] 1) Enumerate all subsets: {1, 2, 3}, {1, 2}, {2, 3}, {1, 3}, {1}, {2}, {3}, empty set.

[0264] 2) Add each company to the subset that does not include itself and calculate their respective marginal contributions, assuming that:

[0265] N={1,2,3},

[0266] v({1})=0, v({2})=0, v({3})=0,

[0267] v({1,2})=90, v({1,3})=80, v({2,3})=70,

[0268] v({1,2,3})=120

[0269] v(empty set) = 0, where v is the valuation function val; v{1} can be understood as, when there is only company 1, the income is 0 yuan; v{2, 3} can be understood as, when only companies 2 and 3 cooperate, the income is 700,000 yuan.

[0270] Then for company 1, according to the above formula (7), add company 1 to the subsets that do not include itself, and calculate the marginal contribution (also known as marginal benefit) in each feature subset. The specific calculation method is as follows:

[0271] Company 1 joins the subset {2, 3}, and its marginal contribution is: 2 / 6*{v{1,2,3}-v{2,3}}=100 / 6;

[0272] Company 1 joins subset {3}, and its marginal contribution is: 1 / 6*{v{1,3}-v{3}}=80 / 6;

[0273] Company 1 joins subset {2}, and its marginal contribution is: 1 / 6*{v{1,2}-v{2}}=90 / 6;

[0274] Company 1 is added to the empty set, and its marginal contribution is: 0;

[0275] Therefore, the total contribution value (Shapely value) of Company 1 to the total profit of 1.2 million is: 270 / 6=45 million.

[0276] Similarly, the above formula can be used to calculate that the contributions of Company 2 and Company 3 are 40W and 35W.

[0277] The SHAP value in the embodiment of the present application can be understood as a variant of the Shapely value, which is used to measure the contribution of a certain feature value to the prediction result (model output result), specifically:

[0278] Assume that there is a machine learning model f, and the input sample X contains features {1, 2, 3, ..., m}, where the feature is replaced by the subscript i. There are M features in total, N is a subset, S is a feature subset that does not contain i, and f_x is an estimation function that returns the average output of model f under the feature subset. The SHAP value calculation formula is the same as the Shapely value, with only different signs, as follows:

[0279]

[0280] The calculation of SHAP value is explained below. For example, suppose there is a sample x = {age = 20, height = 170, income = 100}, and the machine learning model f predicts that the score of f for this sample is 1.2.

[0281] 1) Enumerate all feature subsets: {age=20, height=170, income=100}, {age=20, height=170}, {age=20, income=100}, {height=170, income=100}, {income=100}, {age=20}, {height=170}, {}.

[0282] 2) Traverse each feature in sample x, add each feature value to the subset that does not contain itself, calculate their respective marginal contributions, and use the valuation function f_x to calculate the prediction results corresponding to each feature subset. Assume that:

[0283] f_x({age=20, height=170, income=100})=1.2;

[0284] f_x({age=20, height=170})=0.9;

[0285] f_x({age=20, income=100})=0.8;

[0286] f_x({height=170, income=100})=0.7;

[0287] f_x({income=100})=0;

[0288] f_x({age=20})=0;

[0289] f_x({height=170})=0;

[0290] f_x({})=0;

[0291] It should be noted that when the feature subset is the full set of features in sample x, the prediction result is calculated using the valuation function f_x.

[0292] Next, the contribution of the feature value {age = 20} to the predicted result value 1.2 of the sample is calculated, and {age = 20} is added to the feature subset that does not contain itself, and the marginal contribution value is calculated according to the above formula (6), as follows:

[0293] {Age = 20} is added to the feature subset {Height = 170, Income = 100}, and the marginal contribution is: 2 / 6*{1.2-0.7}=1 / 6;

[0294] {Age = 20} is added to the feature subset {Income = 100}, and the marginal contribution is: 1 / 6*{0.8-0}=0.8 / 6;

[0295] {age = 20} is added to the feature subset {age = 20}, and the marginal contribution is: 1 / 6*{0.9-0}=0.9 / 6;

[0296] {age=20} is added to the empty set, and the marginal contribution is: 0;

[0297] The total contribution is 2.7 / 6=0.45;

[0298] Therefore, the feature value {age=20} contributes 0.45 to the final prediction score of 1.2.

[0299] Continuing to explain the above estimation function f_x, when a sample is given and the SHAP value of each feature value is evaluated, it is necessary to determine a valuation method f_x and use this valuation method to correctly reflect the average value of the model output (prediction score) under a certain feature subset. The following common solutions are provided:

[0300] Taking the sample {age=20, height=170, income=100} as an example, assuming there is a machine learning model f, the longitudinal federated learning model in the embodiment of the present application is:

[0301] Solution (1): If a feature is missing in the subset, the mean / mode / median of the feature value is used instead.

[0302] Assume that there is a feature subset {age = 20, height = 170}. Obviously, without the feature "income", the machine learning model f cannot predict the sample. In practical applications, the feature subset {age = 20, height = 170} can be filled with the mean / mode / median corresponding to the feature "income" (which can be calculated from the training data) to obtain the filled sample {age = 20, height = 170, income = T}, where T is the feature value mean / mode / median corresponding to the feature income, and the predicted value of the filled sample is used as an estimate of the model output average.

[0303] Assume that from the training sample set, the average value of the "income" feature is calculated to be 50, the average value of the "height" feature is calculated to be 150, and the average value of the "age" feature is calculated to be 15. Then: after filling the feature subset {age = 20, height = 170}, the filled sample obtained is {age = 20, height = 170, income = 50}, and

[0304] f_x({age=20, height=170})=f({age=20, height=170, income=50})

[0305] For another example, suppose there is a feature subset {age = 20}, and the feature subset is filled to obtain the filled sample {age = 20, height = 150, income = 50}, and

[0306] f_x({age=20})=f({age=20, height=150, income=50}),

[0307] In addition, if it is an empty set, the output value of f_x is the average of the predicted scores of all training samples.

[0308] Solution (2): If a feature is missing in the subset, it is directly replaced by 0.

[0309] For example, assuming that there is a feature subset {age = 20}, the sample is padded with 0 to obtain a padded sample {age = 20, height = 0, income = 0}, and

[0310] f_x({age=20})=f({age=20, height=0, income=0}).

[0311] Solution (3): If a feature is missing in the feature subset, multiple sampling is performed within the range of possible values ​​of the missing feature to obtain multiple synthetic samples. The average of the prediction scores of the machine learning model for the multiple synthetic samples is the output value of f_x.

[0312] Assume that the age range is 10-30 and the height range is 150-180

[0313] Calculate the predicted value f_x({income=100}) of the feature subset {income=100},

[0314] Suppose that 5 samples are constructed, and the missing features age and height in the feature subset are randomly sampled, and the constructed samples s1 = {age = 15, height = 160, income = 100}, s2 = {age = 21, height = 164, income = 100}, s3 = {age = 29, height = 175, income = 100}, s4 = {age = 25, height = 155, income = 100}, s5 = {age = 28, height = 180, income = 100},

[0315] f_x({income=100})=[f(s1)+f(s2)+f(s3)+f(s4)+f(s5)] / 5.

[0316] Based on the explanation of the above SHAP-related knowledge, in actual federated learning scenarios, calculating the SHAP value of a feature using the above formula (7) or formula (8) often requires a very high amount of calculation, which places high demands on the computing power of electronic devices (servers or terminals), increasing the complexity of actual deployment.

[0317] Generally speaking, in actual federated learning scenarios, the guest side will have one to dozens of dimensions of features, while the host side will generally have hundreds to thousands of dimensions of features. According to the above scheme, all the features of the host are regarded as one feature, and then all subsets are enumerated according to the SHAP / Shapely calculation process. The number of subsets will also be huge. For example, the guest side provides 20-dimensional features, and a host side regards all the features of the host side as one feature. To explain any sample, the number of subsets that need to be fully enumerated is 2 20+1 =2097152. If 100 samples need to be explained, the number of samples for communication / prediction is 209715200, which is a very large amount of calculation.

[0318] Based on this, subsets are sampled according to a certain strategy, and weighted linear regression is used to estimate the SHAP value to greatly reduce the computational overhead.

[0319] For example, a simple 0, 1 vector is used to represent the feature subset, and h is the mapping function:

[0320] h({age=20, height=170, income=100})=[1,1,1],

[0321] h({age=20, height=170})=[1,1,0],

[0322] h({age=20, income=100})=[1,0,1],

[0323] h({height=170, income=100})=[0,1,1],

[0324] h({income=100})=[1,0,0],

[0325] h({age=20})=[0,1,0],

[0326] h({height=170})=[0,0,1],

[0327] h({})=[0,0,0],

[0328] From the above description, we can see that each subset will have a corresponding valuation result:

[0329] f_x({age=20, height=170, income=100})=1.2,

[0330] f_x({age=20, height=170})=0.9,

[0331] f_x({age=20, income=100})=0.8,

[0332] f_x({height=170, income=100})=0.7,

[0333] f_x({income=100})=0,

[0334] f_x({age=20})=0,

[0335] f_x({height=170})=0,

[0336] f_x({})=0,

[0337] In actual implementation, we can use 0 and 1 vectors as X, the output of the f_x function as Y, and assign a certain weight to each vector according to the size of the subset (generally smaller and larger subsets have higher weights). We can then train a linear regression model. The weight of the linear regression model is the SHAP value estimated for each feature.

[0338] In the federated learning scenario, the above method is used to estimate the importance of guest features and participating host features, and finally calculate the contribution value.

[0339] Next, we will explain the specific process of estimating the SHAP value of a feature by combining the characteristics of SHAP and training a linear regression model in the vertical federated learning scenario.

[0340] To execute the data processing method of the federated learning model provided in the embodiment of the present application (the longitudinal federated machine learning model interpretation method based on SHAP value), it is necessary to set a longitudinal federated learning model f, and the input information of f is as follows:

[0341] The training data set (training samples) and samples to be explained are X0, X1, ..., Xn, a total of N items, where Guest and Host each hold some features. Capital G_i (i is an integer) represents a feature provided by the Guest (such as age), and capital H_i (i is an integer) represents a feature provided by the Host. For example, the Guest provides G_1, ..., G_n, a total of n features; the Host provides H_1, ..., H_m, a total of m features; lowercase g_j (j is an integer) represents the feature value corresponding to the feature provided by the Guest (such as age = 20), such as g_1, ..., g_n; lowercase h_j (j is an integer) represents the feature value corresponding to the feature provided by the Host, such as h_1, ..., h_m. If there are multiple hosts, a tag is added at the end to distinguish each host, such as h_1_1, h_1_2, host_1_3 represent the first feature of Host_1, the first feature of Host_2, and the first feature of Host_3 respectively. If there are multiple hosts, the number of features held by the k-th host is m_k, and host_m_k represents the m-th feature of the k-th Host.

[0342] In order to protect the privacy of the host's features on the guest's side, the host's features are represented anonymously on the guest's side. It can be a randomly generated character, or a method agreed upon between the guest and the host. Here, the symbol a is used to represent the anonymity of the host at the guest's side. For example, a_1_1 is the anonymity of h_1_1 at the guest's side.

[0343] Estimation of the set value of the function f_x method: Use one of the three methods mentioned above.

[0344] Sampling times sample_n: If the f_x method uses the third method mentioned above, this parameter takes effect and represents the number of random sampling feature values.

[0345] Subset sampling number max_subset: A fixed value, usually 2048, the number of subsets sampled when using linear regression to estimate SHAP values.

[0346] The participants in the embodiment of the present application include: the Guest party (which may be called the first participant or the leading party), the Host party (which may be called the second participant or the participant). In addition, there may be multiple Host parties.

[0347] Based on the aforementioned input information, the functional modules of the system for implementing the longitudinal federated machine learning model interpretation method based on SHAP value provided in the embodiment of the present application are: subset sampling module, sample interpretation module and prediction module. Fig.13 , Fig.13 is a flow chart of the data processing method of the federated learning model provided in the embodiment of the present application, combined with Fig.13 Describe the overall process of the data processing method of the federated learning model.

[0348] In step 601, the first participant counts the number of first features and the number of second participants participating in modeling the federated learning model, and receives anonymous features corresponding to the second features returned by the second participants.

[0349] In actual implementation, for the sake of distinction, the features provided by the Guest can be called the first features, and the features provided by the Host can be called the second features. Assume that there are k (k ≥ 1 and k is an integer) Hosts participating in the modeling of the federated tree model. The Guest treats all the features of the Host (H_0…H_m) as anonymous features, that is, the Guest does not know the specific names of each feature of the Host. Anonymous (abbreviated as a) can be used to represent the anonymous features corresponding to the features of the Host, such as a_0_0, a_0_1, a_0_2 respectively represent the first feature of Host_0 (the first host), the first feature of Host_1, and the first feature of Host_2. The Guest has features G_0, G_1, …G_n, a_0_0, a_1_0, a_2_0, …a_(m_k)_k, a total of n+m features. In addition, the first participant sends an anonymous feature acquisition request to each second participant, so that the second participant returns the anonymous feature corresponding to the second feature according to the anonymous feature acquisition request.

[0350] In step 602, the first participant inputs the feature subset consisting of each first feature and each anonymous feature into a subset sampling module, and obtains a preset number of subset sampling results according to a preset subset sampling strategy.

[0351] In actual implementation, the sample to be explained (i.e., a single training sample used as a federated learning model) can be regarded as containing a total of n+k features (n first features and k anonymous features), and the final total number of samples is pre-set, denoted as max_subset.

[0352] The result of subset sampling can be regarded as a matrix with max_subset rows and (n+k) columns. The guest retains this matrix as the training matrix. The guest copies the first n columns of the training matrix, which is called the guest subset matrix, and uses the last k columns as the host subset matrix.

[0353] See also Fig.14 , Fig.14 This is a schematic diagram of a subset matrix provided by an embodiment of the present application. In the figure, the Host subset matrix is ​​divided into multiple matrices according to m_0, m_1, ...m_k, and these subset matrices are sent to each corresponding Host. The Guest saves the corresponding weight, which is called the training weight.

[0354] Exemplarily, assuming that there are 5 features of a Guest party and 3 Host parties, Host1, Host2 and Host3, where Host1 provides 2 features (m1=2), Host2 provides 1 feature (m2=1), and Host3 provides 3 features (m2=3), it can be determined that the Host party subset matrix includes 6 dimensions (6 columns, arranged as Host1->Host2->Host3), the first 2 columns are sent to Host1, the last three columns are sent to Host3, and the remaining columns are sent to Host2.

[0355] max_subset represents the number of subset samples, which can be understood as the number of training samples used for the linear regression model. For example, max_subset = 10, a training sample used as a machine learning model f, where the guest provides 4 features, that is, n = 4, and only one host participates, that is, k = 1. The subset sampling result is a matrix of max_subset*(n+k), which is a 10x5 matrix (10 rows and 5 columns), where the last column comes from the host and the first 4 columns come from the guest.

[0356] In step 603, the first participant inputs each sample to be interpreted into the sample interpretation module to obtain the contribution value of each feature in the sample.

[0357] The guest side obtains the contribution value of each feature (first feature) provided by each guest side and the contribution of each anonymous feature of each host side from the feature contribution calculation module. The host side does not obtain any results.

[0358] The guest adds up the contribution values ​​of the features [G_0, G_1, ..., G_n] corresponding to all samples X0, X1, ..., Xn to be explained and the absolute value of the contribution value of each anonymous feature of each sample to be explained to obtain the feature importance of each feature.

[0359] Since the prediction result of the sample is sensitive information, it is safer for the Host to obtain a calculated ratio value. The Guest adds the importance of all features (including the anonymous features of the Host) to obtain a summary value, and then divides the importance of each anonymous feature of the Host by this summary value to obtain a ratio value, and sends the ratio value of each anonymous feature to the corresponding Host, which maps it to the second feature corresponding to each anonymous feature.

[0360] Finally, the Guest outputs the above results, and the Host outputs the contribution ratio of each feature of its own side.

[0361] Next, the specific functions implemented by the subset sampling module are described in detail. The subset sampling module samples the subset according to a certain weight strategy and saves the results. The module input is the total number of features, and the preset subset sampling number max_subset is assumed. Assume that the total number of features is M, and M is a positive integer greater than 1. Fig.15 , Fig.15 This is a schematic diagram of the subset sampling process provided in the embodiment of the present application, based on Fig.13 , step 602 can be performed by Fig.15 The specific steps shown are implemented. Fig.15 The steps shown illustrate the subset sampling process.

[0362] Step 701: The first participant device enumerates the sizes of all possible feature subsets.

[0363] From 1 to M-1 (ignoring the empty set and the full set), taking M=5 as an example, the number of subsets with one element in the subset is C(1,5) which is 5, the number of subsets with two elements in the subset is C(2,5), the number of subsets with three elements in the subset is C(3,5), and the number of subsets with four elements in the subset is C(4,5). The total number of subsets is: Total=C(1,5)+C(2,5)+C(3,5)+C(4,5)=5+10+10+5=30.

[0364] Step 702, weighted scoring is performed on subsets of different sizes to obtain a scoring vector, wherein different sizes refer to the number of elements in the subset.

[0365] The subsets are weighted and scored using the aforementioned weight scoring formula (3) W(m) = (M-1) / (m*(Mm)). Finally, all feature subsets of sizes from 1 to M-1 are scored, and a scoring vector weight is obtained:

[0366] weight=[w_1,……,w_(M-1)], where w_1 represents the weight score of the subset with a subset size of 1.

[0367] Taking M=5 as an example, all subsets of sizes from 1 to 4 are scored using the above formula (3): the scoring vector weight=[1, 4 / 6, 4 / 6, 1] is obtained.

[0368] Step 703, calculate the proportion of the weight of each feature subset to the total according to the scoring vector to obtain a proportion vector.

[0369] Here, the proportional vector is denoted as p, and the above formula (5) is used:

[0370] p=weight / (w_1+……+w_(M-1))=[p1,p2,……,p_(M-1)], and the proportional vector corresponding to the weight vector is obtained, and the value range of each element in the proportional vector is 0 to 1.

[0371] For example, taking M=5, through step 503, weight=[1,4 / 6,4 / 6,1] is obtained, thereby obtaining the p value corresponding to the weight: p_1=1 / (1+4 / 6+4 / 6+1)=0.3, p_2=1 / 5=0.2, p_3=0.2, p_4=0.3, and obtaining the proportional vector p=[0.3,0.2,0.2,0.3].

[0372] Step 704, sort the proportion vectors from large to small to obtain sorted proportion vectors.

[0373] From the properties of W(m) in formula (3), we can see that the weights of larger and smaller subsets are larger, and the weights of complementary sets are equal.

[0374] For example, taking M=5, W(1)=W(4), W(2)=W(3), so the sorted value is p=[p_1, p_(M-1), p_2, p_(M-2)], that is, p=[p_1, p_4, p_2, p_3]=[0.3, 0.3, 0.2, 0.2].

[0375] Step 705, determine the capacity corresponding to the current sampling round.

[0376] It should be noted that the proportional coefficient corresponding to the current round is recorded as p_max, and the corresponding subset size is p_size. p_max is multiplied by the total number of samples required to be sampled in the current round, i_subset, where i represents the sampling round, and when i=1, i.e. the first round of sampling, i_subset=max_subset, and a capacity number is obtained: capacity=i_subset*p_max.

[0377] For example, taking M=5 as an example, preset max_subset=20, and obtain the sorted proportion vector: p=[p_1, p_4, p_2, p_3]=[0.3, 0.3, 0.2, 0.2]. If taking the first round of sampling as an example, p_max=p_1, p_size=1, and obtain a capacity number capacity=0.3*20=6.

[0378] Step 706, compare whether the current capacity number is greater than or equal to the number of feature subsets corresponding to the current round, if so, go to step 707, otherwise, go to step 709.

[0379] Here, the capacity obtained in step 705 is compared with C(p_size, M), and different operations are performed according to the comparison result.

[0380] Step 707: Add the vector representation corresponding to the feature subset of the corresponding size of the current sampling round to the vector result.

[0381] For example, taking M=5 as an example, in the first round of sampling, when p_size=1, the vectors [1,0,0,0,0], [0,1,0,0,0], [0,0,1,0,0], [0,0,0,1,0], [0,0,0,0,1] are added to the vector result; the weight calculation formula for each vector in the linear regression model is: w_(p_size) / C(p_size,M),

[0382] Take M = 5, p_size = 1, and a vector with a subset size of 1 as an example. The weight in the linear regression model is w_1 / C(1,5) = 1 / 5. Add the corresponding weight of each vector to the weight list w_list. Update the value of i_subset to: i_subset = max_subset-C(p_size,M) for the next round of subset selection.

[0383] Step 708, update the scoring vector, determine whether the elements in the scoring vector are empty, if so, execute step 709, otherwise execute step 703.

[0384] Here, w_(p_size) is removed from the aforementioned score vector weight vector list. If weight is finally empty, go to step 709, otherwise return to step 703, that is, loop execution to start the next round of subset selection operation.

[0385] Step 709: When the remaining current sampling number max_subset cannot further accommodate the complete subset, random sampling is performed on the remaining current sampling number max_subset.

[0386] Here, the current sampling number is the updated max_subset, i.e., i_subset. Calculate p for the remaining weights in weight, p is used as the probability of each size subset being selected, then randomly sample max_subset subset sizes, and then randomly generate a 0, 1 vector based on the subset size, add it to the result, and its corresponding weight size is sum(weight) / max_subset, i.e., the remaining weight divided by the remaining number of samples, and execute step 707.

[0387] Step 710: All subset sampling is completed, and the vector result and the corresponding weight list are returned.

[0388] For example, see Fig. 9 The vector results shown should be noted that the vector results can be regarded as a vector matrix composed of 0 and 1. Each row is mapped from the feature subset obtained by sampling. The mapping rule is that the missing features in the subset are represented by 0, and the normal features in the subset are represented by 1. The total number of rows in the vector matrix represents the total number of sampled samples.

[0389] Next, the function of the sample interpretation module is described. The sample interpretation module takes a sample X to be interpreted as input. Fig.16 , Fig.16 This is a sample explanation flow diagram provided in the embodiment of the present application, based on Fig.13 , step 603 can be done by Fig.16 The specific steps shown are implemented. Fig.16 The steps shown illustrate a sample interpretation process.

[0390] Step 801: The first participant and the second participant each input the characteristic part of the sample to be interpreted into the prediction module to obtain a prediction value as a reference value.

[0391] It should be noted that if it is a multi-classification, the predicted value is a q-dimensional vector. The sample to be explained X is the sample used as the federated learning model. Check the setting value of the f_x method. If it is the aforementioned method 1, the Guest or Host calculates the mean, median or mode of each feature based on the training data as the filling value; if it is the aforementioned method 2, 0 is used as the filling value; if it is the aforementioned method 3, the Guest and Host count the possible values ​​of each feature for random sampling.

[0392] Step 802: The first participant loads its own subset matrix, and the second participant loads its own subset matrix, generating the same number of samples to be predicted that are used as the federated learning model.

[0393] Here, the Guest generates the corresponding subset according to the 0,1 feature vector of each row in the subset matrix, and then generates the Guest sample to be predicted according to the Guest-side feature (first feature) of the subset and the sample to be explained X. Similarly, the Host generates the corresponding subset according to the 0,1 feature vector of each row in the subset matrix, and then generates the Host sample to be predicted according to the Host-side feature (second feature) of the subset and the sample to be explained X.

[0394] At this point, the guest and multiple hosts will generate the same number of corresponding samples to be predicted for f_x. The guest and host will input the generated samples into the prediction module, perform federated prediction, and obtain the scoring vector for each sample as y. If it is a multi-classification task, each sample corresponds to a p-dimensional prediction result, and the result is a scoring matrix Y.

[0395] In step 803, the first participant and the second participant respectively input the samples to be predicted generated by themselves into the prediction module, perform federated prediction, and obtain the prediction result of the federated learning model corresponding to each sample to be predicted.

[0396] Here, the prediction result of the federated learning model corresponding to the sample can also be called the sample's scoring vector, denoted as y, where each item in y is minus the reference value. If it is a multi-classification task, the prediction result of each sample corresponding to a p-dimensional matrix can also be called the sample's scoring matrix, denoted as Y, where each item in Y is minus the reference value.

[0397] In step 804, the first participant uses the training matrix as a sample, the training weights as weights, and the predicted values ​​of the corresponding federated learning model of the sample to be predicted as labels to train a linear regression model.

[0398] In actual implementation, if it is a binary classification task or regression task, the guest uses the training matrix as the sample, the training weight as the weight, and y as the label to train a linear regression model (without truncation, i.e. bias). The first n weights of the model correspond to the contribution of each feature of the guest (the first feature), and the last m items correspond to the contribution of each host feature (in this case, the anonymous features corresponding to each second feature). The weights are extracted and returned as the explanation results; otherwise, it is a multi-classification task, the scoring matrix Y is split into l scoring vectors y, and the above process is repeated p times to obtain p results, corresponding to the contribution of each feature to the prediction result of each label.

[0399] Finally, the prediction module is explained. The prediction module takes specific samples as input. The Guest and Host load the vertical federated machine learning model f, predict the input samples based on f, and return the predicted value.

[0400] The optimized longitudinal federated learning model interpretation method provided in the embodiment of the present application can determine the feature contribution of each feature provided by the Guest in a single sample, and measure the feature contribution of the Host. The same is true for multiple hosts, which makes up for the deficiencies in the federated scenario. In addition, it also retains the same measurement of global feature importance as the traditional method. The optimization of subset sampling greatly reduces the complexity of communication and calculation, making it practical to interpret multiple samples and measure contribution functions in the federated scenario.

[0401] The following further describes an exemplary structure of the data processing device 555 of the federated learning model provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2A As shown, Figure 2A The diagram shows a schematic diagram of the structure of the first participant device 400 provided in an embodiment of the present application. The software modules in the data processing device 555 of the federated tree model stored in the memory 540 may include:

[0402] An acquisition module 5551 is used to acquire at least one first feature provided by the first participant device for training the federated learning model;

[0403] A receiving module 5552, configured to receive at least one anonymous feature sent by a second participant device, wherein the anonymous feature corresponds to a second feature used to train the federated learning model;

[0404] A selection module 5553 is used to select at least one feature from the first feature and the anonymous feature, and obtain a prediction result of each feature corresponding to the federated learning model;

[0405] A training module 5554 is used to construct a training sample of the linear model based on the at least one feature, and train the linear model based on the training sample and the prediction result to obtain a model parameter when the linear model converges;

[0406] The determination module 5555 is used to determine, based on the model parameters, first contribution information of each of the first features corresponding to the prediction result, and second contribution information of each of the anonymous features corresponding to the prediction result.

[0407] In some embodiments, the receiving module is also used to send an anonymous feature acquisition request carrying a sample identifier to the second participant device; wherein the sample identifier is used to identify the sample to which the first feature belongs for training the federated learning model, and the anonymous feature acquisition request is used for the second participant device to respond to the anonymous feature acquisition request, determine the second feature corresponding to the sample identifier, and the anonymous feature corresponding to the second feature; and receive the anonymous feature corresponding to the sample identifier returned by the second participant device.

[0408] In some embodiments, the receiving module is used to sum the first contribution information of each of the first features corresponding to the prediction result and the second contribution information of each of the anonymous features corresponding to the prediction result to obtain aggregated contribution information; respectively obtain the ratio of each of the second contribution information to the aggregated contribution information, and send the ratio to the second participant device, so that the second participant device determines the contribution information of the second feature corresponding to each of the anonymous features based on the ratio.

[0409] In some embodiments, the selection module is also used to construct a feature set including the at least one first feature and at least one anonymous feature; obtain a feature subset included in the feature set; sample a target number of target feature subsets from multiple feature subsets, and use the features included in the target feature subset as the at least one feature selected.

[0410] In some embodiments, the selection module is also used to obtain the weight score of each of the feature subsets; sort the feature subsets in descending order of the weight score to obtain a feature subset sequence; and sample sequentially starting from the first feature subset based on the feature subset sequence to obtain a target number of feature subsets as target feature subsets.

[0411] In some embodiments, the training module is also used to obtain the conversion relationship between the feature and the training sample of the linear model; based on the conversion relationship, the at least one feature is converted respectively to obtain the vector representation corresponding to each feature; and the vector representation is used as the training sample of the linear model.

[0412] In some embodiments, the training module is further used to use the prediction results as sample labels of the corresponding training samples to train the linear model and obtain model parameters when the linear model converges.

[0413] In some embodiments, Figure 2B As shown, Figure 2B The diagram shows a schematic diagram of the structure of the second participant device 410 provided in an embodiment of the present application. The software modules stored in the data processing device 555 of the federated tree model of the memory 540 may include:

[0414] A generating module 5556, configured to generate, for each second feature used to train the federated learning model, anonymous features corresponding to each second feature;

[0415] The sending module 5557 is used to send the anonymous features corresponding to each of the second features to the first participant device; wherein the anonymous features are used for the first participant device to select at least one feature from at least one first feature used to train the federated learning model and the anonymous features, to construct a training sample of the linear model, and based on the training sample and the prediction result of the federated learning model corresponding to the at least one feature, to train the linear model to obtain the model parameters of the linear model when convergence occurs, and based on the model parameters, to determine the first contribution information of each of the first features corresponding to the prediction result, and the second contribution information of the anonymous features corresponding to the prediction result.

[0416] In some embodiments, the generation module is also used to receive an anonymous feature acquisition request carrying a sample identifier sent by the first participant device; parse the anonymous feature acquisition request to obtain the sample identifier; determine the second feature corresponding to the sample identifier and the anonymous feature corresponding to the second feature; and send the anonymous feature to the first participant device.

[0417] In some embodiments, the sending module is used to receive the ratio corresponding to the anonymous feature sent by the first participant device, and the ratio is used to characterize the contribution ratio of the anonymous feature to the prediction result; based on the ratio, determine the corresponding contribution information of the second feature, and output the contribution information.

[0418] It should be noted that the description of the device of the embodiment of the present application is similar to the description of the above-mentioned method embodiment, and has similar beneficial effects as the method embodiment, so it will not be repeated.

[0419] An embodiment of the present application provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, the data processing method of the federated learning model provided in the embodiment of the present application is implemented.

[0420] The present application embodiment provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will be caused to execute the method provided by the present application embodiment, for example, Figure 3 The data processing method of the federated learning model is shown.

[0421] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0422] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0423] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0424] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0425] In summary, the embodiments of the present application can not only expand the scope of application in federated learning scenarios, but also accurately measure the contribution information of each feature in a single training sample and measure the contribution information of the second participant, while greatly reducing the computational complexity of the model and improving computational efficiency.

[0426] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A data processing method for a federated learning model, characterized in that: Applied to a first participant device, the first participant device at least includes a laptop, a tablet computer, a desktop computer, a smart phone, a dedicated messaging device, a portable gaming device, a smart speaker, and a smart watch, the method comprising: Acquire at least one first feature provided by the first participant device for training the federated learning model, where the first feature includes age and education level; Receive at least one anonymous feature sent by a second participant device, where the anonymous feature corresponds to a second feature used to train the federated learning model, where the second feature includes income and work, and the anonymous feature is determined by hashing each second feature used to train the federated learning model by the second participant device, and the hash value corresponding to each second feature is obtained, where the second participant device includes at least a laptop, a tablet computer, a desktop computer, a smart phone, a dedicated messaging device, a portable gaming device, a smart speaker, and a smart watch; Selecting at least one feature from the first feature and the anonymous feature, and obtaining a prediction result of the federated learning model corresponding to each feature; Based on the at least one feature, construct a training sample of the linear model, and based on the training sample and the prediction result, train the linear model to obtain model parameters when the linear model converges; Based on the model parameters, determining first contribution information of each of the first features corresponding to the prediction result, and second contribution information of each of the anonymous features corresponding to the prediction result; After determining, based on the model parameters, first contribution information of each of the first features corresponding to the prediction result, and second contribution information of each of the anonymous features corresponding to the prediction result, the method includes: Sum up the first contribution information of each of the first features corresponding to the prediction result and the second contribution information of each of the anonymous features corresponding to the prediction result to obtain summary contribution information; The ratio of each second contribution information to the aggregated contribution information is obtained respectively, and the ratio is sent to the second participant device, so that the second participant device determines the contribution information of the second feature corresponding to each anonymous feature based on the ratio, and the contribution information is used to measure the contribution of the second feature to the prediction result.

2. The method according to claim 1, characterized in that The receiving at least one anonymous feature sent by the second participant device includes: Sending an anonymous feature acquisition request carrying a sample identifier to the second participant device; The sample identifier is used to identify the sample to which the first feature belongs for training the federated learning model, and the anonymous feature acquisition request is used for the second participant device to respond to the anonymous feature acquisition request to determine the second feature corresponding to the sample identifier and the anonymous feature corresponding to the second feature; An anonymous feature corresponding to the sample identifier is received and returned by the second participant device.

3. The method according to claim 1, characterized in that The selecting at least one feature from the first feature and the anonymous feature comprises: constructing a feature set including the at least one first feature and at least one anonymous feature; Acquire a feature subset included in the feature set; A target number of target feature subsets are obtained by sampling from the plurality of feature subsets, and features included in the target feature subsets are used as the at least one feature selected.

4. The method according to claim 3, characterized in that The step of sampling a target number of target feature subsets from a plurality of the feature subsets comprises: Obtaining a weight score for each of the feature subsets; Sorting the feature subsets in descending order of weight scores to obtain a feature subset sequence; According to the feature subset sequence, sequential sampling is performed starting from the first feature subset to obtain a target number of feature subsets as target feature subsets.

5. The method according to claim 1, characterized in that The step of constructing a training sample of a linear model based on the at least one feature includes: Obtaining a conversion relationship between the feature and the training sample of the linear model; Based on the conversion relationship, the at least one feature is converted respectively to obtain a vector representation corresponding to each feature; The vector representation is used as a training sample of the linear model.

6. The method according to claim 1, characterized in that The step of training the linear model based on the training sample and the prediction result to obtain model parameters when the linear model converges includes: The prediction results are used as sample labels of the corresponding training samples to train the linear model, and the model parameters of the linear model when it converges are obtained.

7. A data processing method for a federated learning model, characterized in that: Applied to a second participant device, the second participant device at least includes a laptop, a tablet computer, a desktop computer, a smart phone, a dedicated messaging device, a portable gaming device, a smart speaker, and a smart watch, the method comprising: For each second feature used to train the federated learning model, hash processing is performed respectively to obtain a hash value corresponding to each second feature, and the hash value corresponding to each second feature is used as an anonymous feature corresponding to each second feature, where the second feature includes income and work; Sending the anonymous feature corresponding to each of the second features to a first party device, wherein the first party device includes at least a laptop, a tablet computer, a desktop computer, a smart phone, a dedicated messaging device, a portable gaming device, a smart speaker, and a smart watch; The anonymous feature is used for the first participant device to select at least one feature from at least one first feature used for training the federated learning model and the anonymous feature, construct a training sample of the linear model, and train the linear model based on the training sample and the prediction result of the federated learning model corresponding to the at least one feature to obtain the model parameters when the linear model converges, and determine the first contribution information of each first feature corresponding to the prediction result and the second contribution information of the anonymous feature corresponding to the prediction result based on the model parameters, wherein the first feature includes age and education level; After sending the anonymous features corresponding to each of the second features to the first participant device, the method further includes: receiving a ratio corresponding to the anonymous feature sent by the first participant device, where the ratio is used to represent a contribution ratio of the anonymous feature to a prediction result; Based on the ratio, the corresponding contribution information of the second feature is determined and output, where the contribution information is used to measure the contribution of the second feature to the prediction result.

8. The method according to claim 7, characterized in that The sending the anonymous features corresponding to each of the second features to the first participant device includes: Receiving an anonymous feature acquisition request carrying a sample identifier sent by the first participant device; Parsing the anonymous feature acquisition request to obtain the sample identifier; Determine the second feature corresponding to the sample identifier and an anonymous feature corresponding to the second feature; The anonymous characteristic is sent to the first party device.

9. A data processing device for a federated learning model, characterized in that: include: an acquisition module, configured to acquire a first feature for a federated learning model provided by a first participant device, wherein the first participant device includes at least a laptop, a tablet computer, a desktop computer, a smart phone, a dedicated messaging device, a portable gaming device, a smart speaker, and a smart watch, and the first feature includes age and education level; A receiving module, configured to receive at least one anonymous feature sent by a second participant device, wherein the anonymous feature corresponds to a second feature used to train the federated learning model, and the anonymous feature is determined by hashing each second feature used to train the federated learning model by the second participant device, and the hash value corresponding to each second feature obtained, wherein the second feature includes income and work, and the second participant device includes at least a laptop computer, a tablet computer, a desktop computer, a smart phone, a dedicated messaging device, a portable gaming device, a smart speaker, and a smart watch; A selection module, configured to select at least one feature from the first feature and the anonymous feature, and obtain a prediction result of the federated learning model corresponding to the at least one feature; A training module, configured to construct a training sample of the linear model based on the at least one feature, and to train the linear model based on the training sample and the prediction result to obtain model parameters when the linear model converges; A determination module, configured to determine, based on the model parameters, first contribution information of each of the first features corresponding to the prediction result, and second contribution information of each of the anonymous features corresponding to the prediction result; The receiving module is further used to sum the first contribution information of each of the first features corresponding to the prediction result and the second contribution information of each of the anonymous features corresponding to the prediction result to obtain aggregated contribution information; respectively obtain the ratio of each of the second contribution information to the aggregated contribution information, and send the ratio to the second participant device, so that the second participant device determines the contribution information of the second feature corresponding to each of the anonymous features based on the ratio, and the contribution information is used to measure the contribution of the second feature to the prediction result.

10. An electronic device, characterized in that: include: A memory for storing executable instructions; A processor, used to implement the data processing method of the federated learning model described in any one of claims 1 to 8 when executing the executable instructions stored in the memory.

11. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to implement the data processing method of the federated learning model described in any one of claims 1 to 8 when executed by a processor.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the data processing method of the federated learning model described in any one of claims 1 to 8 is implemented.

Images