Data processing method, device, system and equipment for cross-feature federated learning

Abstract

The embodiments of the present disclosure provide a data processing method, device and equipment for cross-feature federated learning, which relates to the technical field of federated learning. In a cross-feature federated learning scenario, based on the local data of each participant corresponding to different fields, the labeled participant can send the identifier of the sampled data to the unlabeled participant, so that the unlabeled participant can sample based on the identifier of the data, so that the sample data between the participants corresponds to the same data identifier, and then, each participant can also perform numerical processing on the local sample data, thereby realizing a data preprocessing solution combined with federated learning, so that federated learning can be performed between multiple participants.

Description

Technical Field

[0001] The present disclosure relates to the technical field of federated learning, and in particular to a data processing method, apparatus and device for cross-feature federated learning. Background Art

[0002] Federated learning refers to a machine learning framework that can effectively help multiple participants (which can represent individuals or institutions) jointly train models while meeting data privacy protection requirements.

[0003] Normally, each participant will update the parameters of the federated model based on the global parameters federated by the server. However, this process generally refers to the model training phase. Before entering the model training phase, each participant often needs to process their local data to make the local data meet the preset standards so that it can be directly applied to model training.

[0004] However, in the cross-feature federated learning scenario, the sample fields of each participant are inconsistent, and federated learning among multiple participants cannot be achieved based on inconsistent sample fields. Therefore, data preprocessing is required before model training. How to achieve data preprocessing in the cross-feature federated learning scenario so that federated learning among multiple participants can be performed has become an urgent problem to be solved. Summary of the invention

[0005] The purpose of the embodiments of the present disclosure is to provide a data processing method for cross-feature federated learning, a data processing device for cross-feature federated learning, a computer-readable medium and an electronic device, which can realize data preprocessing in the cross-feature federated learning scenario, thereby facilitating federated learning among multiple participants.

[0006] A first aspect of an embodiment of the present disclosure provides a data processing method applied to cross-feature federated learning, the method including a first type of participant and a second type of participant, the first type of participant includes a data label, and the second type of participant does not include a data label, including:

[0007] The first type of participants determine the first type of reference data from the first type of original data according to a preset ratio, and synchronize the identifier corresponding to the first type of reference data to the second type of participants;

[0008] The second type of participants determine the second type of reference data from the second type of original data according to the identifier corresponding to the first type of reference data. The first type of original data corresponds to the first type of participants, and the second type of original data corresponds to the second type of participants.

[0009] The first type of participants samples the first type of reference data to obtain the first type of sample data, and synchronizes the identifier corresponding to the first type of sample data to the second type of participants;

[0010] The second-category participant samples the second-category reference data according to the identifier corresponding to the first-category sample data to obtain the second-category sample data;

[0011] If a type of sample data contains non-numerical data, a type of participant performs numerical processing on the type of sample data and performs model training based on the processed type of sample data;

[0012] The second-category participant digitizes the second-category sample data when the second-category sample data contains non-numerical data, and performs model training based on the processed second-category sample data.

[0013] In an exemplary embodiment of the present disclosure, a type of participant performs numerical processing on a type of sample data, including:

[0014] A type of participant generates a set of statistical values ​​corresponding to a type of sample data; wherein the set of statistical values ​​includes a statistical value corresponding to at least one field;

[0015] A type of participant performs numerical processing on a type of sample data according to a set of statistical values ​​of the type of sample data.

[0016] In an exemplary embodiment of the present disclosure, a type of participant generates a set of statistical values ​​corresponding to a type of sample data, including:

[0017] A type of participant determines the total sample data volume corresponding to a type of sample data;

[0018] The first-class participants determine the sub-data corresponding to the same field in each class of sample data, and obtain the sub-data set corresponding to each field;

[0019] A type of participant calculates a statistical value corresponding to each sub-data set; wherein the statistical value of the sub-data set includes at least one of the first type of sub-data volume, the second type of sub-data volume, the mean, and the mode;

[0020] A type of participant determines a set of statistical values ​​of a type of sample data according to the statistical values ​​of each sub-data set and the total sample data volume.

[0021] In an exemplary embodiment of the present disclosure, a type of participant performs numerical processing on a type of sample data according to a set of statistical values ​​of the type of sample data, including:

[0022] A type of participant determines the proportion of the first type of sub-data and the proportion of the second type of sub-data in the sub-data set;

[0023] If it is detected that the sum of the proportion of the first type of sub-data and the proportion of the second type of sub-data is less than the preset ratio threshold, the first type of participant deletes the sub-data set;

[0024] If it is detected that the proportion of the first type of sub-data is greater than or equal to the proportion of the second type of sub-data, a type of participant converts the second type of sub-data in the sub-data set into first numerical data based on a statistical value set of a type of sample data;

[0025] If it is detected that the proportion of the first type of sub-data is less than the proportion of the second type of sub-data, a type of participant converts the first type of sub-data in the sub-data set into non-numeric data based on a statistical value set of a type of sample data, and uniformly converts the non-numeric data and the second type of sub-data in the sub-data set into second numerical data.

[0026] In an exemplary embodiment of the present disclosure, a type of participant samples a type of reference data to obtain a type of sample data, including:

[0027] A type of participant samples a type of reference data through a preset sampling method to obtain a type of sample data; wherein the preset sampling method includes upsampling, downsampling or stratified sampling.

[0028] In an exemplary embodiment of the present disclosure, a type of participant determines a type of reference data from a type of original data according to a preset ratio, including:

[0029] A type of participant sorts a type of original data according to a preset random number to obtain a sorting result;

[0030] A class of participants determines a class of reference data from the sorting results according to a preset ratio.

[0031] In an exemplary embodiment of the present disclosure, the method further includes:

[0032] A type of participant determines the verification set data from a type of original data, verifies the trained model based on the verification set data, obtains the verification result and outputs it.

[0033] A second aspect of an embodiment of the present disclosure provides a data processing device applied to cross-feature federated learning, including:

[0034] A data selection unit, used to determine a type of reference data from a type of original data according to a preset ratio;

[0035] A data synchronization unit, used to synchronize the identifier corresponding to the first type of reference data to the second type of participants that do not contain data tags; wherein the second type of participants determine the second type of reference data from the second type of original data according to the identifier corresponding to the first type of reference data, the first type of original data corresponds to the first type of participants, and the second type of original data corresponds to the second type of participants;

[0036] The data selection unit is also used to sample a class of reference data to obtain a class of sample data;

[0037] The data synchronization unit is further used to synchronize the identifier corresponding to the first type of sample data to the second type of participants; wherein the second type of participants samples the second type of reference data according to the identifier corresponding to the first type of sample data to obtain the second type of sample data;

[0038] The data application unit is used for digitizing the first category of sample data if the first category of sample data contains non-numerical data, and performing model training based on the processed first category of sample data; wherein the second category of participants digitizes the second category of sample data when the second category of sample data contains non-numerical data, and performs model training based on the processed second category of sample data.

[0039] In an exemplary embodiment of the present disclosure, a data application unit performs numerical processing on a type of sample data, including:

[0040] Generate a statistical value set corresponding to a type of sample data; wherein the statistical value set includes a statistical value corresponding to at least one field;

[0041] A class of sample data is numerically processed according to a set of statistical values ​​of the class of sample data.

[0042] In an exemplary embodiment of the present disclosure, a data application unit generates a set of statistical values ​​corresponding to a type of sample data, including:

[0043] Determine the total sample data volume corresponding to a category of sample data;

[0044] Determine the sub-data corresponding to the same field in each type of sample data, and obtain the sub-data set corresponding to each field;

[0045] Calculating a statistical value corresponding to each sub-data set; wherein the statistical value of the sub-data set includes at least one of the amount of first-type sub-data, the amount of second-type sub-data, a mean, and a mode;

[0046] A set of statistical values ​​of a class of sample data is determined based on the statistical values ​​of each sub-data set and the total sample data volume.

[0047] In an exemplary embodiment of the present disclosure, a data application unit performs numerical processing on a class of sample data according to a statistical value set of the class of sample data, including:

[0048] Determine the proportion of the first type of sub-data and the proportion of the second type of sub-data in the sub-data set;

[0049] If it is detected that the sum of the proportion of the first type of sub-data and the proportion of the second type of sub-data is less than a preset ratio threshold, the sub-data set is deleted;

[0050] If it is detected that the proportion of the first type of sub-data is greater than or equal to the proportion of the second type of sub-data, converting the second type of sub-data in the sub-data set into first numerical data based on a statistical value set of a type of sample data;

[0051] If it is detected that the proportion of the first type of sub-data is less than that of the second type of sub-data, the first type of sub-data in the sub-data set is converted into non-numeric data based on a statistical value set of a type of sample data, and the non-numeric data and the second type of sub-data in the sub-data set are uniformly converted into second numerical data.

[0052] In an exemplary embodiment of the present disclosure, a data selection unit samples a type of reference data to obtain a type of sample data, including:

[0053] A class of reference data is sampled by a preset sampling method to obtain a class of sample data; wherein the preset sampling method includes upsampling, downsampling or stratified sampling.

[0054] In an exemplary embodiment of the present disclosure, the data selection unit determines a type of reference data from a type of original data according to a preset ratio, including:

[0055] Sort a class of original data according to a preset random number to obtain a sorting result;

[0056] A category of reference data is determined from the sorting results according to a preset ratio.

[0057] In an exemplary embodiment of the present disclosure, the data selection unit is also used to determine the verification set data from a type of original data, and verify the trained model according to the verification set data to obtain and output the verification result.

[0058] A third aspect of the embodiments of the present disclosure provides a sample data processing system applied to cross-feature federated learning, the system comprising: a first type of participants including data labels and a second type of participants not including data labels, wherein:

[0059] A first type of participant is used to determine a first type of reference data from a first type of original data according to a preset ratio, and synchronize the identifier corresponding to the first type of reference data to a second type of participant, and the first type of original data corresponds to the first type of participant;

[0060] The second type of participants are used to determine the second type of reference data from the second type of original data according to the identifier corresponding to the first type of reference data, and the second type of original data corresponds to the second type of participants;

[0061] The first type of participants are also used to sample the first type of reference data to obtain the first type of sample data, and synchronize the identifier corresponding to the first type of sample data to the second type of participants;

[0062] The second type of participants are also used to sample the second type of reference data according to the identifiers corresponding to the first type of sample data to obtain the second type of sample data;

[0063] The first type of participants is also used to perform numerical processing on the first type of sample data when the first type of sample data contains non-numerical data, and to perform model training based on the processed first type of sample data;

[0064] The second type of participants are also used to digitize the second type of sample data when the second type of sample data contains non-numerical data, and to perform model training based on the processed second type of sample data.

[0065] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the data processing method applied to cross-feature federated learning as described in the first aspect of the above embodiment is implemented.

[0066] According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method applied to cross-feature federated learning as described in the first aspect of the above embodiment.

[0067] According to a sixth aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0068] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0069] In the technical solutions provided by some embodiments of the present disclosure, a type of reference data can be determined from a type of original data according to a preset ratio, and the identifier corresponding to the type of reference data can be synchronized to a type of participant that does not contain a data label; the type of participant determines the type of reference data from the type of original data according to the identifier corresponding to the type of reference data, the type of original data corresponds to the type of participant, and the type of original data corresponds to the type of participant; the type of reference data is sampled to obtain type of sample data, and the identifier corresponding to the type of sample data is synchronized to the type of participant; the type of participant samples the type of reference data according to the identifier corresponding to the type of sample data to obtain type of sample data; if the type of sample data contains non-numerical data, the type of sample data is numerically processed, and model training is performed based on the processed type of sample data; when the type of sample data contains non-numerical data, the type of participant numerically processes the type of sample data, and model training is performed based on the processed type of sample data.

[0070] In implementing the embodiments of the present disclosure, on the one hand, in a cross-feature federated learning scenario, based on the local data of each participant corresponding to different fields, the labeled participant can send the identifier of the sampled data to the unlabeled participant, so that the unlabeled participant can sample based on the identifier of the data, so that the sample data between the participants corresponds to the same data identifier, and further, each participant can also perform numerical processing on the local sample data, thereby realizing a data preprocessing solution combined with federated learning, so as to perform federated learning among multiple participants.

[0071] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0073] Figure 1 A schematic diagram schematically shows an exemplary system architecture of a data processing method and device for cross-feature federated learning to which an embodiment of the present disclosure can be applied;

[0074] Figure 2 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure is shown;

[0075] Figure 3A flowchart of a data processing method applied to cross-feature federated learning according to an embodiment of the present disclosure is schematically shown;

[0076] Figure 4 A flowchart of a data processing method applied to cross-feature federated learning according to another embodiment of the present disclosure is schematically shown;

[0077] Figure 5 A sequence diagram schematically illustrates a data processing system applied to cross-feature federated learning according to an embodiment of the present disclosure;

[0078] Figure 6 The structural block diagram of a data processing device applied to cross-feature federated learning according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0079] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0080] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0081] Figure 1 A schematic diagram schematically shows an exemplary system architecture of a data processing method and device for cross-feature federated learning to which an embodiment of the present disclosure can be applied.

[0082] like Figure 1As shown, the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The terminal devices 101, 102, 103 may be various electronic devices with display screens, including but not limited to desktop computers, portable computers, smart phones, tablet computers, etc. It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only illustrative. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers. For example, the terminal devices 101, 102 or 103 corresponding to the first type of participants may execute: determining a type of reference data from a type of original data according to a preset ratio, and synchronizing the identifier corresponding to the type of reference data to the second type of participants that do not contain data tags. Furthermore, the terminal devices 101, 102 or 103 corresponding to the second type of participants may execute: determining the second type of reference data from the second type of original data according to the identifier corresponding to the first type of reference data. Furthermore, the terminal devices 101, 102 or 103 corresponding to the first type of participants may execute: sampling the first type of reference data to obtain the first type of sample data, and synchronizing the identifier corresponding to the first type of sample data to the second type of participants. Furthermore, the terminal devices 101, 102 or 103 corresponding to the second type of participants may execute: sampling the second type of reference data according to the identifier corresponding to the first type of sample data to obtain the second type of sample data. Furthermore, the terminal device 101, 102 or 103 corresponding to the first type of participants can execute: if the first type of sample data contains non-numeric data, the first type of sample data is numerically processed, and the model training is performed based on the processed first type of sample data. Furthermore, the terminal device 101, 102 or 103 corresponding to the second type of participants can execute: if the second type of sample data contains non-numeric data, the second type of sample data is numerically processed, and the model training is performed based on the processed second type of sample data.

[0083] Figure 2 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure is shown.

[0084] It should be noted that Figure 2 The computer system 200 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0085] like Figure 2As shown, the computer system 200 includes a central processing unit (CPU) 201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 202 or the program loaded from the storage part 208 to the random access memory (RAM) 203. In the (RAM) 203, various programs and data required for system operation are also stored. The (CPU) 201, the (ROM) 202 and the (RAM) 203 are connected to each other through a bus 204. An input / output (I / O) interface 205 is also connected to the bus 204.

[0086] The following components are connected to the (I / O) interface 205: an input section 206 including a keyboard, a mouse, etc.; an output section 207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 208 including a hard disk, etc.; and a communication section 209 including a network interface card such as a LAN card, a modem, etc. The communication section 209 performs communication processing via a network such as the Internet. A drive 210 is also connected to the (I / O) interface 205 as needed. A removable medium 211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 210 as needed, so that a computer program read therefrom is installed into the storage section 208 as needed.

[0087] In particular, according to an embodiment of the present disclosure, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 209, and / or installed from a removable medium 211. When the computer program is executed by the central processing unit (CPU) 201, various functions defined in the method and apparatus of the present application are executed.

[0088] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0089] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0090] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0091] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being installed in the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the following embodiment. For example, the electronic device may implement the following method. Figure 3 The various steps shown, etc.

[0092] This example embodiment provides a data processing method for cross-feature federated learning, the method including a first type of participant and a second type of participant, the first type of participant includes a data label, and the second type of participant does not include a data label. Figure 3 As shown, the data processing method applied to cross-feature federated learning may include the following steps S310 to S360, specifically:

[0093] Step S310: A first type of participant determines a first type of reference data from a first type of original data according to a preset ratio, and synchronizes an identifier corresponding to the first type of reference data to a second type of participant.

[0094] Step S320: The second type of participant determines the second type of reference data from the second type of original data according to the identifier corresponding to the first type of reference data. The first type of original data corresponds to the first type of participant, and the second type of original data corresponds to the second type of participant.

[0095] Step S330: a first type of participant samples a first type of reference data to obtain a first type of sample data, and synchronizes an identifier corresponding to the first type of sample data to a second type of participant.

[0096] Step S340: The second category of participants samples the second category of reference data according to the identifier corresponding to the first category of sample data to obtain the second category of sample data.

[0097] Step S350: If a type of sample data includes non-numerical data, a type of participant performs numerical processing on the type of sample data, and performs model training based on the processed type of sample data.

[0098] Step S360: The second-category participant performs numerical processing on the second-category sample data when the second-category sample data contains non-numerical data, and performs model training based on the processed second-category sample data.

[0099] Among them, steps S310 to S330 can be executed by the terminal device / server corresponding to any participant in the cross-feature federated learning scenario, and this application does not limit the number of participants in the cross-feature federated learning scenario. The cross-feature federated learning scenario includes a type of participant (guest) and a type of participant (host). In actual applications, if the data of the participant contains a data label, the participant can be determined as a type of participant (guest), otherwise the participant is determined as a type of participant (host).

[0100] Among them, the first category of original data is the local data of the first category of participants (guest), and the second category of original data is the local data of the second category of participants (host). The fields corresponding to the first category of original data and the second category of original data are different. The first category of original data contains more fields than the second category of original data. The first category of original data contains label fields, and the second category of original data does not contain label fields. The first category of reference data is the data obtained by the first category of participants (guest) after initially screening the first category of original data, and the second category of reference data is the data obtained by the second category of participants (host) after initially screening the second category of original data based on the identifier corresponding to the first category of reference data. The first category of sample data is the data obtained by the first category of participants (guest) sampling the first category of reference data, and the second category of sample data is the data obtained by the second category of participants (host) sampling the second category of reference data based on the identifier corresponding to the first category of sample data.

[0101] Implementation Figure 3 The data processing method shown is applied to cross-feature federated learning. In the cross-feature federated learning scenario, based on the local data of each participant corresponding to different fields, the labeled participant (i.e., the first type of participant) can send the identifier of the sampled data to the unlabeled participant (i.e., the second type of participant), so that the unlabeled participant can sample based on the identifier of the data, so that the sample data between the participants correspond to the same data identifier. Then, each participant can also perform numerical processing on the local sample data, thereby realizing a data preprocessing scheme combined with federated learning, so as to conduct federated learning between multiple participants. In addition, the sample data can be de-characterized by numerical processing of non-numeric data, which is conducive to further training of the federated model.

[0102] Next, the above steps of this exemplary embodiment are described in more detail.

[0103] In step S310, a first type of participant determines a first type of reference data from a first type of original data according to a preset ratio, and synchronizes an identifier corresponding to the first type of reference data to a second type of participant.

[0104] Specifically, a data tag can be understood as an attribute identifier / type identifier for a type of original data, and a data tag can be represented as a numerical value, a letter, a string, a text, etc., which is not limited in the embodiments of the present application. In addition, the number of the first type of participants and the number of the second type of participants are not limited in the embodiments of the present application. The identifier corresponding to the first type of reference data can be represented as a string, a numerical value, etc., which is not limited in the embodiments of the present application.

[0105] For example, the original data of one type of participants can be expressed as the following Table 1, and the original data of two types of participants can be expressed as the following Table 2.

[0106] User ID Name gender age Frequency of violations 1 ** male 20 0 2 *** female 43 3 3 ## female Fourty 5 4 ### male 21 1 5 $$$ male 32 0

[0107] Table 1

[0108] User ID Name gender age 1 ** male 20 2 *** female 43 3 ## female Fourty 4 ### male 21 5 $$$ male 32

[0109] Table 2

[0110] Before step S310, the first category of original data of the first category of participants is represented as Table 3 below, and the second category of original data of the second category of participants is represented as Table 4 below. The above method may further include the following steps: aligning the first category of original data and the second category of original data according to preset rules, Table 3 can be represented as Table 1 after alignment, and Table 4 can be represented as Table 2 after alignment; wherein the preset rules are used to limit the conditions for alignment, such as rearranging the first category of original data and the second category of original data according to the numerical order of user IDs.

[0111]

[0112]

[0113] Table 3

[0114] User ID Name gender age 3 ## female Fourty 1 ** male 20 4 ### male 21 2 *** female 43 5 $$$ male 32

[0115] Table 4

[0116] In step S320, the second type of participant determines the second type of reference data from the second type of original data according to the identifier corresponding to the first type of reference data. The first type of original data corresponds to the first type of participant, and the second type of original data corresponds to the second type of participant.

[0117] The method in which the second type of participants determine the second type of reference data from the second type of original data according to the identifier corresponding to the first type of reference data can be: the second type of participants select data corresponding to the same identifier from the second type of original data as the second type of reference data according to the identifier corresponding to the first type of reference data. The first type of reference data can be one or more. If there are multiple first type of reference data, each first type of reference data corresponds to an identifier, that is, multiple identifiers, and the second type of reference data selected according to the multiple identifiers can also be multiple.

[0118] For example, the first type of reference data determined from the first type of original data shown in Table 1 according to the preset ratio can be shown in Table 5, and the second type of reference data selected by the second type of participants from the second type of original data shown in Table 2 according to the identifiers (e.g., user ID) in Table 5 can be shown in Table 6:

[0119] User ID Name gender age Frequency of violations 1 ** male 20 0 2 *** female 43 3 3 ## female Fourty 5

[0120] Table 5

[0121]

[0122]

[0123] Table 6

[0124] As an optional embodiment, determining a type of reference data from a type of original data according to a preset ratio includes: sorting a type of original data according to a preset random number to obtain a sorting result; and determining a type of reference data from the sorting result according to the preset ratio.

[0125] Specifically, the preset random number may be a random number generated according to a random number generation algorithm, and the random numbers held by the first and second participants are the same. The preset ratio may be expressed as a numerical value, a ratio value, a percentage, etc., which is not limited in the embodiments of the present application.

[0126] It can be seen that by implementing this optional embodiment, the randomness of data selection can be improved by sorting a type of original data, which is beneficial to improving the effect of subsequent model training. In addition, a part of the data in the sorting result can be selected as a type of reference data based on a preset ratio, thereby reducing the amount of data transmission between other participants and improving the efficiency of data preprocessing.

[0127] In step S330, a first type of participant samples a first type of reference data to obtain a first type of sample data, and synchronizes an identifier corresponding to the first type of sample data to a second type of participant.

[0128] Specifically, the first category sample data obtained by sampling the first category reference data shown in Table 5 can be expressed as the following Table 7, and then the identifier corresponding to the first category sample data in Table 7 can be synchronized to the second category participants, so that the second category participants can sample the second category sample data consistent with the identifier corresponding to the first category sample data from the second category reference data shown in Table 6 according to the identifier corresponding to the first category sample data. The second category sample data can be expressed as the following Table 8.

[0129] User ID Name gender age Frequency of violations 2 *** female 43 3

[0130] Table 7

[0131] User ID Name gender age 2 *** female 43

[0132] Table 8

[0133] As an optional embodiment, sampling a class of reference data to obtain a class of sample data includes: sampling a class of reference data using a preset sampling method to obtain a class of sample data; wherein the preset sampling method includes upsampling, downsampling or stratified sampling.

[0134] Specifically, the preset sampling method may be other sampling methods in addition to the above-mentioned sampling method, which is not limited in the embodiments of the present application. In addition, a class of reference data is sampled by upsampling to obtain a class of sample data, including: determining the value range of each field information in the target field, each field information comes from a different class of reference data, and the target field can be any field in the fields corresponding to the class of reference data; determining the number of field information contained in each value range, sampling a class of reference data based on the value range corresponding to the maximum number of field information, and obtaining a class of sample data; wherein the number of a class of sample data is consistent with the number of field information in a specific value range, and the specific value range contains the minimum number of field information.

[0135] For example, a first-class reference data may include field information corresponding to the fields "user ID", "name", "gender", "age", "violation frequency", and "overdue frequency", namely, "2", "***", "female", "43", "3", and "0". The target field may be the above-mentioned field "age", and the value range of each field information in the target field may include [1-10, 10-20, 30-40, 40-100], and the field information in the target field may be "1, 4, 10, 12, 15, 20, 40, 50, 55" respectively. The value range [1-10] includes field information "1" and "4", that is, the number of field information corresponding to the value range [1-10] is 2, and the field information "1" and "4" belong to different types of reference data respectively; the value range [1-10] includes field information "10", "12", "15", and "20", that is, the number of field information corresponding to the value range [10-20] is 4, and the field information "10", "12", "15", and "20" belong to different types of reference data respectively; the value range [30-40] includes field information "40", that is, the number of field information corresponding to the value range [1-10] is 1; the value range [40-100] includes field information "50" and "55", that is, the number of field information corresponding to the value range [40-100] is 2, and the field information "50" and "55" belong to different types of reference data respectively. It can be seen that the number of field information corresponding to the value range [1-10, 10-20, 30-40, 40-100] is different, that is, there is a sample imbalance and the number of field information corresponding to the value range [10-20] is the largest. Then, a class of sample data can be obtained by sampling from a class of reference data to which the field information corresponding to the value range [10-20] belongs. The number of the class of sample data can be 1, which corresponds to the value range containing the least number of field information (e.g., 1), thereby achieving upsampling.

[0136] In addition, a class of reference data is sampled by downsampling to obtain a class of sample data, including: determining the value range of each field information in the target field, each field information comes from a different class of reference data, and the target field can be any field in the fields corresponding to the class of reference data; sampling a class of reference data according to the number of field information in each value range to obtain a class of sample data, and the proportion of the class of sample data contained in each value range is consistent with a preset ratio (e.g., 1:1:1).

[0137] Based on the above example, when the number of field information corresponding to the value range [1-10, 10-20, 30-40, 40-100] is different, the same number of field information is obtained from each value range and a class of reference data corresponding to the field information is determined as a class of sample data.

[0138] In addition, a class of reference data is sampled by stratified sampling to obtain a class of sample data, including: determining the value corresponding to the class of reference data under each field; determining a preset value range set corresponding to each field; selecting intermediate data for each field based on the preset value range set, and for this field, the values ​​of the intermediate data fall within each preset value range in the preset value range set (such as [10-20, 20-30, 30-40]); determining the union of the intermediate data corresponding to each field, and determining the data in the union as the above-mentioned class of sample data, so that balanced values ​​can be achieved for a class of reference data to train a more accurate model.

[0139] In another method, a class of reference data is sampled by stratified sampling to obtain a class of sample data, including: determining a specific field from each field, wherein the specific field may be a field corresponding to the most field information; determining a set of preset value ranges corresponding to the specific field; and selecting a class of sample data based on the preset value range set, wherein the values ​​of the class of sample data respectively fall within each preset value range in the preset value range set (e.g., [10-20, 20-30, 30-40]).

[0140] It can be seen that by implementing this optional embodiment, a class of sample data that is more suitable for model training can be sampled from a class of reference data in a balanced manner, and the accuracy of the model can be improved based on the effective class of sample data.

[0141] In step S340, the second type of participants samples the second type of reference data according to the identifiers corresponding to the first type of sample data to obtain the second type of sample data.

[0142] Specifically, the second category of participants can sample data corresponding one-to-one with the identifier from the second category of reference data according to the identifier corresponding to the first category of sample data as the second category of sample data.

[0143] In step S350, if a type of sample data includes non-numerical data, a type of participant performs numerical processing on the type of sample data, and performs model training based on the processed type of sample data.

[0144] Specifically, non-numeric data may include letter data, string data, etc., which are not limited in the embodiments of the present application. If the number of sample data of a category is greater than or equal to 1, the step is executed: a category of participants performs numerical processing on the sample data of a category. If the number of sample data of a category is less than 1, this step is not executed.

[0145] In addition, model training is performed based on the processed sample data of a class, including: if the number of processed sample data of a class is 1, the processed sample data of a class is input into the federated model of a class of participants, and the loss function between the result of the model output and the preset result (such as label) corresponding to the sample data of a class is calculated, and then the model parameters are adjusted according to the loss function, so as to update the federated model of a class of participants; if the number of processed sample data of a class is greater than 1, the processed multiple sample data of a class are input into the model in sequence to realize multiple training of the federated model of a class of participants, wherein the model is the federated model of the participants, each participant corresponds to one or more federated models, and the model architecture can be the same or different between different participants. Among them, the loss function can be an absolute value loss function, a log loss function, a square loss function, an exponential loss function, a Hinge loss function, a perceptual loss function, a cross entropy loss function, etc., which is not limited in the embodiments of the present application.

[0146] In addition, model training is performed based on the processed sample data, including: performing feature derivation and / or feature selection based on the processed sample data to obtain feature engineering results, and performing model training based on the feature engineering results.

[0147] In step S360, the second-category participant performs numerical processing on the second-category sample data when the second-category sample data contains non-numerical data, and performs model training based on the processed second-category sample data.

[0148] Among them, when the second category of participants contains non-numerical data in the second category of sample data, the second category of sample data is digitized, and the model training is performed based on the processed second category of sample data. If the number of processed second category of sample data is 1, the processed second category of sample data is input into the federated model of the second category of participants, and the loss function between the result of the model output and the preset result (such as label) corresponding to the second category of sample data is calculated, and then the model parameters are adjusted according to the loss function, so as to realize the update of the federated model of the second category of participants; if the number of processed second category of sample data is greater than 1, multiple processed second category of sample data are input into the federated model of the second category of participants in sequence to realize multiple training of the federated model of the second category of participants.

[0149] In an exemplary embodiment of the present disclosure, the above method also includes: determining verification set data from a type of original data, and verifying the trained model based on the verification set data to obtain and output the verification results.

[0150] Specifically, determining the validation set data from a category of original data includes: determining the data other than the reference data in the category of original data as the validation set data; or determining the validation set data from the category of original data according to the validation set selection ratio (e.g., 50%). The validation set data determined in this way may or may not overlap with the sample data.

[0151] In addition, the trained model is verified according to the verification set data to obtain and output the verification results, including: inputting the verification set data into the trained model in sequence to obtain the verification results corresponding to each data in the verification set data, and the verification results can be expressed as a string, text, numerical value, etc., which is not limited to the embodiments of the present application; the verification results are structured to obtain a structured representation of the verification results (such as a table), and the structured representation is output, which can facilitate users to intuitively understand the verification results and improve the intuitiveness of the output verification results.

[0152] In addition, the second type of participants can also determine the verification set data from the second type of original data, and verify the trained federated model based on the verification set data to obtain and output the verification results.

[0153] It can be seen that by implementing this optional embodiment, verification set data can be divided to verify the trained model, and the verification results used to characterize the model training accuracy can be displayed to the user so that the user can understand the application effect of the model trained with numerical sample data.

[0154] As an optional embodiment, numerical processing is performed on a class of sample data, including: generating a statistical value set corresponding to the class of sample data; wherein the statistical value set includes statistical values ​​corresponding to at least one field; and numerical processing is performed on the class of sample data according to the statistical value set of the class of sample data.

[0155] Specifically, it also includes: the way in which the second category of participants perform numerical processing on the second category of sample data can be: generating a statistical value set corresponding to the second category of sample data; wherein the statistical value set includes statistical values ​​corresponding to at least one field; and performing numerical processing on the second category of sample data according to the statistical value set of the second category of sample data.

[0156] In addition, if a class of sample data does not contain non-numeric data, it also includes: generating a statistical value set corresponding to the class of sample data; wherein the statistical value set includes statistical values ​​corresponding to at least one field; and numerating the class of sample data according to the statistical value set of the class of sample data.

[0157] Among them, the statistical value set of the second category sample data can be the same as, different from, or partially the same as the statistical value set of the first category sample data. For example, the statistical value set of the first category sample data can be expressed as [median 192.33, mode male, mean 23.66], and the statistical value set of the second category sample data can be expressed as [median 56, mode female, mean 23.66].

[0158] It can be seen that by implementing this optional embodiment, it is possible to realize numerical processing of data based on a set of statistical values, so that the processed data meets the preset standards that can be used for model training.

[0159] As an optional embodiment, generating a set of statistical values ​​corresponding to a category of sample data includes: determining the total sample data volume corresponding to the category of sample data; determining the sub-data corresponding to the same field in each category of sample data to obtain a sub-data set corresponding to each field; calculating the statistical value corresponding to each sub-data set; wherein the statistical value of the sub-data set includes at least one of the first type of sub-data volume, the second type of sub-data volume, the mean and the mode; and determining the set of statistical values ​​of the category of sample data based on the statistical value of each sub-data set and the total sample data volume.

[0160] Specifically, if a category of sample data is represented as the above-mentioned Table 7, the total sample data volume corresponding to the category of sample data is 1, and the sub-data corresponding to the same field in each category of sample data may include: a sub-dataset corresponding to the field "user ID" [2], a sub-dataset corresponding to the field "name" [***], a sub-dataset corresponding to the field "gender" [female], a sub-dataset corresponding to the field "age"

[43] , and a sub-dataset corresponding to the field "violation frequency" [3].

[0161] In addition, the second category of participants generates a set of statistical values ​​corresponding to the second category of sample data, including: determining the total sample data volume corresponding to the second category of sample data; determining the sub-data corresponding to the same field in each of the second category of sample data, and obtaining a sub-data set corresponding to each field; calculating the statistical value corresponding to each sub-data set; wherein the statistical value of the sub-data set includes at least one of the second type sub-data volume, the second type sub-data volume, the mean and the mode; determining the statistical value set of the second category of sample data based on the statistical value of each sub-data set and the total sample data volume.

[0162] It can be seen that by implementing this optional embodiment, the sample data can be divided into multiple sub-data sets according to the fields to which they belong, so as to perform efficient and accurate numerical processing based on the fields.

[0163] As an optional embodiment, a class of sample data is numerically processed according to a statistical value set of the class of sample data, including: determining the proportion of first type sub-data and the proportion of second type sub-data in the sub-data set; if it is detected that the sum of the proportion of first type sub-data and the proportion of second type sub-data is less than a preset ratio threshold, deleting the sub-data set; if it is detected that the proportion of first type sub-data is greater than or equal to the proportion of second type sub-data, converting the second type sub-data in the sub-data set to first numerical data based on the statistical value set of the class of sample data; if it is detected that the proportion of first type sub-data is less than the proportion of second type sub-data, converting the first type sub-data in the sub-data set to non-numerical data based on the statistical value set of the class of sample data, and uniformly converting the non-numerical data and the second type sub-data in the sub-data set to second numerical data.

[0164] Specifically, the preset ratio threshold can be expressed as delete_ratio. If a type of sample data is as shown in Table 5, the sub-dataset corresponding to the field "user ID" is [1, 2, 3], the sub-dataset corresponding to the field "name" is [**, ***, ##], the sub-dataset corresponding to the field "gender" is [male, female, female], the sub-dataset corresponding to the field "age" is [20, 43, Fourty], the sub-dataset corresponding to the field "violation frequency" is [0, 3, 5], and the sub-dataset corresponding to the field "overdue frequency" is [2, 0, 1].

[0165] Taking the sub-dataset [**, ***, ##] corresponding to the field "name" as an example, if it is detected that the sum of the proportion of the first type of sub-data and the proportion of the second type of sub-data in the sub-dataset is less than the preset ratio threshold, the sub-dataset is deleted; wherein, the method of deleting the sub-dataset can be: deleting the field information containing the specific field (i.e., name) in each type of sample data, and the type of sample data after deleting the sub-dataset can be expressed as the following Table 8:

[0166] User ID gender age Frequency of violations Overdue frequency 1 male 20 0 2 2 female 43 3 0 3 female Fourty 5 1

[0167] Table 8

[0168] Taking the sub-dataset [20, 43, Fourty] corresponding to the field "age" as an example, if it is detected that the proportion of the first type of sub-data is greater than or equal to the proportion of the second type of sub-data, the second type of sub-data [Fourty] in the sub-dataset is converted into first numerical data (such as the median 31.5) based on the global statistical value set; wherein, the method of converting the second type of sub-data in the sub-dataset into the first numerical data based on the global statistical value set can be: selecting specific parameters from the global statistical value set, and replacing all the second type of sub-data in the sub-dataset with the specific parameters, so that the sub-dataset only contains the first numerical data, and a class of sample data including the sub-dataset after numerical processing can be expressed as the following Table 9; wherein, the specific parameter can be the mode or the median, etc., which is not limited in the embodiments of the present application.

[0169] User ID gender age Frequency of violations Overdue frequency 1 male 20 0 2 2 female 43 3 0 3 female 31.5 5 1

[0170] Table 9

[0171] If it is detected that the proportion of the first type of sub-data is less than the proportion of the second type of sub-data, the first type of sub-data in the sub-data set is converted into non-numeric data based on a global statistical value set (e.g., mode), and the non-numeric data and the second type of sub-data in the sub-data set are uniformly converted into first numerical data; wherein, the non-numeric data and the second type of sub-data in the sub-data set are uniformly converted into the first numerical data, including: the non-numeric data and the second type of sub-data in the sub-data set are uniformly converted into a preset numerical value (e.g., 100), and the preset numerical value belongs to the first numerical data; or, based on a global statistical value set (e.g., a fusion result of feature vectors of each sub-data in the sub-data set), the non-numeric data and the second type of sub-data in the sub-data set are uniformly converted into a preset numerical value.

[0172] In addition, the second category sample data is numerically processed according to the statistical value set of the second category sample data, including: determining the proportion of the first type of sub-data and the proportion of the second type of sub-data in the sub-data set; if it is detected that the sum of the proportion of the first type of sub-data and the proportion of the second type of sub-data is less than a preset ratio threshold, deleting the sub-data set; if it is detected that the proportion of the first type of sub-data is greater than or equal to the proportion of the second type of sub-data, converting the second type of sub-data in the sub-data set into first numerical data based on the statistical value set of the second category sample data; if it is detected that the proportion of the first type of sub-data is less than the proportion of the second type of sub-data, converting the second type of sub-data in the sub-data set into non-numerical data based on the statistical value set of the second category sample data, and uniformly converting the non-numerical data and the second type of sub-data in the sub-data set into second numerical data.

[0173] It can be seen that by implementing this optional embodiment, digitization methods can be implemented in various situations to cope with the situations of different sub-data sets, so as to efficiently and conveniently implement digitization processing for each sub-data set.

[0174] See also Figure 4 , Figure 4 Schematically shows Figure 3 A flowchart of a data processing method for cross-feature federated learning according to another embodiment of the present invention. Figure 4 As shown, the data processing method applied to cross-feature federated learning includes: steps S400 to S424.

[0175] Step S400: sorting a category of original data according to a preset random number to obtain a sorting result, and determining a category of reference data from the sorting result according to a preset ratio.

[0176] Step S402: synchronize the identifier corresponding to the first category of reference data to the second category of participants that do not contain data tags; wherein the second category of participants are used to determine the second category of reference data from the second category of original data according to the identifier corresponding to the first category of reference data, the first category of original data corresponds to the first category of participants, and the second category of original data corresponds to the second category of participants.

[0177] Step S404: sampling a class of reference data using a preset sampling method to obtain a class of sample data; wherein the preset sampling method includes upsampling, downsampling or stratified sampling.

[0178] Step S406: synchronizing the identifier corresponding to the first category of sample data to the second category of participants; wherein the second category of participants are used to sample the second category of reference data according to the identifier corresponding to the first category of sample data to obtain the second category of sample data.

[0179] Step S408: If a type of sample data includes non-numeric data, determine the total sample data volume corresponding to the type of sample data, and determine the sub-data corresponding to the same field in each type of sample data to obtain a sub-data set corresponding to each field.

[0180] Step S410: Calculate the statistical value corresponding to each sub-data set; wherein the statistical value of the sub-data set includes at least one of the first type of sub-data volume, the second type of sub-data volume, the mean, and the mode.

[0181] Step S412: Determine a statistical value set of a category of sample data according to the statistical value of each sub-data set and the total sample data volume.

[0182] Step S414: Determine the proportion of the first type of sub-data and the proportion of the second type of sub-data in the sub-data set that has not been digitized. If it is detected that the sum of the proportion of the first type of sub-data and the proportion of the second type of sub-data is less than the preset ratio threshold, execute step S416. If it is detected that the proportion of the first type of sub-data is greater than or equal to the proportion of the second type of sub-data, execute step S418. If it is detected that the proportion of the first type of sub-data is less than the proportion of the second type of sub-data, execute step S420.

[0183] Step S416: Delete the sub-dataset. Then, execute step S422.

[0184] Step S418: based on the statistical value set of a type of sample data, convert the second type of sub-data in the sub-data set into first numerical data. Then, execute step S422.

[0185] Step S420: based on a set of statistical values ​​of a type of sample data, convert the first type of sub-data in the sub-data set into non-numeric data, and convert the non-numeric data and the second type of sub-data in the sub-data set into second numeric data. Then, execute step S422.

[0186] Step S422: Check whether the numerical processing of all sub-data sets in a type of sample data is completed. If yes, execute step S424. If not, execute step S414.

[0187] Step S424: Perform model training based on the processed first-category sample data; wherein the second-category participant is used to digitize the second-category sample data when the second-category sample data contains non-numerical data, and perform model training based on the processed second-category sample data.

[0188] It should be noted that steps S400 to S424 are Figure 3 The steps and their embodiments shown correspond to each other. For the specific implementation of steps S400 to S424, please refer to Figure 3 The steps and embodiments shown are not described in detail here.

[0189] It can be seen that implementation Figure 4The method shown, in the cross-feature federated learning scenario, based on the local data of each participant corresponding to different fields, the labeled participant can send the identifier of the sampled data to the unlabeled participant, so that the unlabeled participant can sample based on the identifier of the data, so that the sample data between the participants corresponds to the same data identifier, and then, each participant can also perform numerical processing on the local sample data, thereby realizing a data preprocessing solution combined with federated learning, so that federated learning can be performed between multiple participants. In addition, the sample data can be de-characterized by numerical processing of non-numeric data, which is conducive to further training of the federated model.

[0190] See also Figure 5 , Figure 5 A sequence diagram of a data processing system applied to cross-feature federated learning according to an embodiment of the present disclosure is schematically shown. Figure 5 As shown, the data processing system applied to cross-feature federated learning includes: steps S510 to S580.

[0191] Step S510: A type of participant determines a type of reference data from a type of original data according to a preset ratio, and a type of original data corresponds to a type of participant.

[0192] Step S520: The first type of participant synchronizes the identifier corresponding to the first type of reference data to the second type of participant.

[0193] Step S530: The second type of participant determines the second type of reference data from the second type of original data according to the identifier corresponding to the first type of reference data, and the second type of original data corresponds to the second type of participant.

[0194] Step S540: A type of participant samples a type of reference data to obtain a type of sample data.

[0195] Step S550: The first type of participant synchronizes the identifier corresponding to the first type of sample data to the second type of participant.

[0196] Step S560: The second category of participants samples the second category of reference data according to the identifier corresponding to the first category of sample data to obtain the second category of sample data.

[0197] Step S570: When a type of participant includes non-numerical data in a type of sample data, the type of sample data is numerically processed, and model training is performed based on the processed type of sample data.

[0198] Step S580: When the second-category participant includes non-numerical data in the second-category sample data, the second-category sample data is numerically processed, and model training is performed based on the processed second-category sample data.

[0199] Specifically, a first-class participant (guest) can select a first-class reference data from a first-class original data containing a tag field stored locally according to a preset ratio (e.g., 30%), and synchronize the identifier corresponding to the first-class reference data (e.g., 321321, 432421, 53215, etc.) to a second-class participant (host), wherein the number of the second-class participants can be one or more. Furthermore, the second-class participant (host) can select the second-class reference data from the second-class original data not containing a tag field stored locally according to the identifier corresponding to the first-class reference data; wherein the identifier of the second-class reference data corresponds to the identifier of the first-class reference data one-to-one; further, the first-class participant (guest) can sample the first-class reference data to obtain a first-class sample data, wherein the first-class sample data can be understood as a part of the first-class reference data, wherein the sampling method can be random sampling, upsampling, downsampling, stratified sampling, etc. Furthermore, the first type of participant (guest) can synchronize the identifier corresponding to the first type of sample data to the second type of participant (host), so that the second type of participant (host) samples the second type of reference data based on the identifier corresponding to the first type of sample data to obtain the second type of sample data, and the second type of sample data is a part of the second type of reference data. Furthermore, when the first type of sample data contains non-numeric data, the first type of participant (guest) can perform numerical processing on the first type of sample data, and perform model training based on the processed first type of sample data, and when the first type of sample data does not contain non-numeric data, directly perform model training based on the first type of sample data. The second type of participant (host) can perform numerical processing on the second type of sample data, and perform model training based on the processed second type of sample data, and when the second type of sample data does not contain non-numeric data, directly perform model training based on the second type of sample data.

[0200] It should be noted that steps S510 to S580 are Figure 3 The steps and their embodiments shown correspond to each other. For the specific implementation of steps S510 to S580, please refer to Figure 3 The steps and embodiments shown are not described in detail here.

[0201] It can be seen that implementation Figure 5In the system shown, in a cross-feature federated learning scenario, based on the local data of each participant corresponding to different fields, the labeled participant can send the identifier of the sampled data to the unlabeled participant, so that the unlabeled participant can sample based on the identifier of the data, so that the sample data between the participants corresponds to the same data identifier. Furthermore, each participant can also perform numerical processing on the local sample data, thereby realizing a data preprocessing solution combined with federated learning, so that federated learning can be performed between multiple participants. In addition, the sample data can be de-characterized by numerical processing of non-numeric data, which is conducive to further training of the federated model.

[0202] Furthermore, in this example implementation, a data processing device for cross-feature federated learning is also provided. Figure 6 As shown, the data processing device 600 applied to cross-feature federated learning may include:

[0203] The data selection unit 601 is used to determine a type of reference data from a type of original data according to a preset ratio;

[0204] The data synchronization unit 602 is used to synchronize the identifier corresponding to the first type of reference data to the second type of participants that do not contain data tags; wherein the second type of participants are used to determine the second type of reference data from the second type of original data according to the identifier corresponding to the first type of reference data, the first type of original data corresponds to the first type of participants, and the second type of original data corresponds to the second type of participants;

[0205] The data selection unit 601 is further used to sample a type of reference data to obtain a type of sample data;

[0206] The data synchronization unit 602 is further used to synchronize the identifier corresponding to the first type of sample data to the second type of participants; wherein the second type of participants are used to sample the second type of reference data according to the identifier corresponding to the first type of sample data to obtain the second type of sample data;

[0207] The data application unit 603 is used to perform numerical processing on the first category of sample data if the first category of sample data contains non-numerical data, and to perform model training based on the processed first category of sample data; wherein the second category of participants are used to perform numerical processing on the second category of sample data when the second category of sample data contains non-numerical data, and to perform model training based on the processed second category of sample data.

[0208] It can be seen that implementation Figure 6The device shown, in a cross-feature federated learning scenario, based on the local data of each participant corresponding to different fields, the labeled participant can send the identifier of the sampled data to the unlabeled participant, so that the unlabeled participant can sample based on the identifier of the data, so that the sample data between the participants corresponds to the same data identifier, and then, each participant can also perform numerical processing on the local sample data, thereby realizing a data preprocessing solution combined with federated learning, so that federated learning can be performed between multiple participants. In addition, the sample data can be de-characterized by numerical processing of non-numeric data, which is conducive to further training of the federated model.

[0209] In an exemplary embodiment of the present disclosure, the data application unit 603 performs numerical processing on a type of sample data, including:

[0210] Generate a statistical value set corresponding to a type of sample data; wherein the statistical value set includes a statistical value corresponding to at least one field;

[0211] A class of sample data is numerically processed according to a set of statistical values ​​of the class of sample data.

[0212] It can be seen that by implementing this optional embodiment, it is possible to realize numerical processing of data based on a set of statistical values, so that the processed data meets the preset standards that can be used for model training.

[0213] In an exemplary embodiment of the present disclosure, the data application unit 603 generates a set of statistical values ​​corresponding to a type of sample data, including:

[0214] Determine the total sample data volume corresponding to a category of sample data;

[0215] Determine the sub-data corresponding to the same field in each type of sample data, and obtain the sub-data set corresponding to each field;

[0216] Calculating a statistical value corresponding to each sub-data set; wherein the statistical value of the sub-data set includes at least one of the amount of first-type sub-data, the amount of second-type sub-data, a mean, and a mode;

[0217] A set of statistical values ​​of a class of sample data is determined based on the statistical values ​​of each sub-data set and the total sample data volume.

[0218] It can be seen that by implementing this optional embodiment, the sample data can be divided into multiple sub-data sets according to the fields to which they belong, so as to perform efficient and accurate numerical processing based on the fields.

[0219] In an exemplary embodiment of the present disclosure, the data application unit 603 performs numerical processing on a class of sample data according to a set of statistical values ​​of the class of sample data, including:

[0220] Determine the proportion of the first type of sub-data and the proportion of the second type of sub-data in the sub-data set;

[0221] If it is detected that the sum of the proportion of the first type of sub-data and the proportion of the second type of sub-data is less than a preset ratio threshold, the sub-data set is deleted;

[0222] If it is detected that the proportion of the first type of sub-data is greater than or equal to the proportion of the second type of sub-data, converting the second type of sub-data in the sub-data set into first numerical data based on a statistical value set of a type of sample data;

[0223] If it is detected that the proportion of the first type of sub-data is less than that of the second type of sub-data, the first type of sub-data in the sub-data set is converted into non-numeric data based on a statistical value set of a type of sample data, and the non-numeric data and the second type of sub-data in the sub-data set are uniformly converted into second numerical data.

[0224] It can be seen that by implementing this optional embodiment, digitization methods can be implemented in various situations to cope with the situations of different sub-data sets, so as to efficiently and conveniently implement digitization processing for each sub-data set.

[0225] In an exemplary embodiment of the present disclosure, the data selection unit 601 samples a type of reference data to obtain a type of sample data, including:

[0226] A class of reference data is sampled by a preset sampling method to obtain a class of sample data; wherein the preset sampling method includes upsampling, downsampling or stratified sampling.

[0227] It can be seen that by implementing this optional embodiment, a class of sample data that is more suitable for model training can be sampled from a class of reference data in a balanced manner, and the accuracy of the model can be improved based on the effective class of sample data.

[0228] In an exemplary embodiment of the present disclosure, the data selection unit 601 determines a type of reference data from a type of original data according to a preset ratio, including:

[0229] Sort a class of original data according to a preset random number to obtain a sorting result;

[0230] A category of reference data is determined from the sorting results according to a preset ratio.

[0231] It can be seen that by implementing this optional embodiment, the randomness of data selection can be improved by sorting a type of original data, which is beneficial to improving the effect of subsequent model training. In addition, a part of the data in the sorting result can be selected as a type of reference data based on a preset ratio, thereby reducing the amount of data transmission between other participants and improving the efficiency of data preprocessing.

[0232] In an exemplary embodiment of the present disclosure, the data selection unit 601 is also used to determine the verification set data from a type of original data, and verify the trained model according to the verification set data to obtain and output the verification result.

[0233] It can be seen that by implementing this optional embodiment, verification set data can be divided to verify the trained model, and the verification results used to characterize the model training accuracy can be displayed to the user so that the user can understand the application effect of the model trained with numerical sample data.

[0234] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0235] Since the various functional modules of the data processing device applied to cross-feature federated learning in the example embodiment of the present disclosure correspond to the steps of the example embodiment of the data processing method applied to cross-feature federated learning, for details not disclosed in the embodiment of the device of the present disclosure, please refer to the embodiment of the data processing method applied to cross-feature federated learning in the present disclosure.

[0236] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0237] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data processing method for cross-feature federated learning, characterized in that: The method includes a first type of participant and a second type of participant, wherein the first type of participant includes a data tag and the second type of participant does not include the data tag, including: The first type of participants determine a type of reference data from the first type of original data according to a preset ratio, and synchronize the identifier corresponding to the first type of reference data to the second type of participants; the first type of original data is the local data of the first type of participants, and the data tag is the attribute identifier or type identifier of the first type of original data; the first type of original data includes user ID, name, gender, age, violation frequency and overdue frequency; The second type of participants determine the second type of reference data from the second type of original data according to the identifier corresponding to the first type of reference data, the first type of original data corresponds to the first type of participants, and the second type of original data corresponds to the second type of participants; the second type of original data is the local data of the second type of participants, the first type of original data and the second type of original data have different fields, and the first type of original data contains more fields than the second type of original data; the second type of original data includes user ID, name, gender and age; The first type of participants samples the first type of reference data to obtain first type of sample data, and synchronizes identifiers corresponding to the first type of sample data to the second type of participants; The second type of participants samples the second type of reference data according to the identifier corresponding to the first type of sample data to obtain the second type of sample data; If the first type of sample data contains non-numerical data, the first type of participants perform numerical processing on the first type of sample data, and perform model training based on the processed first type of sample data; The second type of participants perform numerical processing on the second type of sample data when the second type of sample data contains non-numerical data, and perform model training based on the processed second type of sample data.

2. The method according to claim 1, characterized in that The first type of participants perform numerical processing on the first type of sample data, including: The first type of participants generates a set of statistical values ​​corresponding to the first type of sample data; wherein the set of statistical values ​​includes a statistical value corresponding to at least one field; The first type of participants performs numerical processing on the first type of sample data according to a set of statistical values ​​of the first type of sample data.

3. The method according to claim 2, characterized in that The first type of participants generates a set of statistical values ​​corresponding to the first type of sample data, including: The first type of participants determines the total sample data volume corresponding to the first type of sample data; The first type of participants determine the sub-data corresponding to the same field in each type of sample data, and obtain the sub-data set corresponding to each field; The first type of participants calculates statistical values ​​corresponding to each of the sub-data sets; wherein the statistical values ​​of the sub-data sets include at least one of the first type of sub-data volume, the second type of sub-data volume, the mean, and the mode; The first type of participants determines a set of statistical values ​​of the first type of sample data according to the statistical values ​​of each of the sub-data sets and the total sample data volume.

4. The method according to claim 3, characterized in that The first type of participants performs numerical processing on the first type of sample data according to the statistical value set of the first type of sample data, including: The first type of participants determines the proportion of the first type of sub-data and the proportion of the second type of sub-data in the sub-data set; If it is detected that the sum of the proportion of the first type of sub-data and the proportion of the second type of sub-data is less than a preset ratio threshold, the first type of participants deletes the sub-data set; If it is detected that the proportion of the first type of sub-data is greater than or equal to the proportion of the second type of sub-data, the first type of participants converts the second type of sub-data in the sub-data set into first numerical data based on the statistical value set of the first type of sample data; If it is detected that the proportion of the first type of sub-data is less than the proportion of the second type of sub-data, the first type of participating party converts the first type of sub-data in the sub-data set into non-numeric data based on the statistical value set of the first type of sample data, and uniformly converts the non-numeric data and the second type of sub-data in the sub-data set into second numerical data.

5. The method according to claim 1, characterized in that The first type of participants samples the first type of reference data to obtain a type of sample data, including: The type of participants samples the type of reference data through a preset sampling method to obtain a type of sample data; wherein the preset sampling method includes upsampling, downsampling or stratified sampling.

6. The method according to any one of claims 1 to 5, characterized in that: The first type of participants determine a type of reference data from a type of original data according to a preset ratio, including: The first type of participants sort the first type of original data according to a preset random number to obtain a sorting result; The type of participants determines a type of reference data from the sorting results according to the preset ratio.

7. The method according to claim 1, characterized in that The method further comprises: The first type of participants determines the verification set data from the first type of original data, verifies the trained model based on the verification set data, obtains the verification result and outputs it.

8. A data processing device applied to cross-feature federated learning, characterized in that: include: A data selection unit, used to determine a type of reference data from a type of original data according to a preset ratio; The first type of original data is local data of a first type of participant, and the first type of participant includes a data tag, and the data tag is an attribute identifier or a type identifier of the first type of original data; the first type of original data includes user ID, name, gender, age, violation frequency and overdue frequency; A data synchronization unit, used for synchronizing the identifier corresponding to the first type of reference data to the second type of participants that do not contain data tags; wherein the second type of participants are used to determine the second type of reference data from the second type of original data according to the identifier corresponding to the first type of reference data, the first type of original data corresponds to the first type of participants, and the second type of original data corresponds to the second type of participants; the second type of original data is the local data of the second type of participants, the first type of original data and the second type of original data have different fields, and the first type of original data contains more fields than the second type of original data; the second type of original data includes user ID, name, gender and age; The data selection unit is further used to sample the reference data to obtain sample data; The data synchronization unit is further used to synchronize the identifier corresponding to the first type of sample data to the second type of participants; wherein the second type of participants are used to sample the second type of reference data according to the identifier corresponding to the first type of sample data to obtain the second type of sample data; The data application unit is used for digitizing the first category of sample data if the first category of sample data contains non-numerical data, and performing model training based on the processed first category of sample data; wherein the second category of participants are used for digitizing the second category of sample data if the second category of sample data contains non-numerical data, and performing model training based on the processed second category of sample data.

9. A sample data processing system applied to cross-feature federated learning, characterized in that: The system includes: a first type of participants including data tags and a second type of participants not including the data tags, wherein: The first type of participants is used to determine a type of reference data from the first type of original data according to a preset ratio, and synchronize the identifier corresponding to the first type of reference data to the second type of participants; the first type of original data is the local data of the first type of participants, and the data tag is the attribute identifier or type identifier of the first type of original data; the first type of original data includes user ID, name, gender, age, violation frequency and overdue frequency; The second type of participants are used to determine the second type of reference data from the second type of original data according to the identifier corresponding to the first type of reference data, the first type of original data corresponds to the first type of participants; the second type of original data is local data of the second type of participants, the first type of original data and the second type of original data have different fields, and the first type of original data contains more fields than the second type of original data; the second type of original data includes user ID, name, gender and age; The first type of participants is further used to sample the first type of reference data to obtain first type of sample data, and synchronize identifiers corresponding to the first type of sample data to the second type of participants; The second type of participants are further used to sample the second type of reference data according to the identifier corresponding to the first type of sample data to obtain the second type of sample data; The first type of participants is further used to perform numerical processing on the first type of sample data when the first type of sample data contains non-numerical data, and perform model training based on the processed first type of sample data; The second type of participants are also used to perform numerical processing on the second type of sample data when the second type of sample data contains non-numerical data, and perform model training based on the processed second type of sample data.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the data processing method for cross-feature federated learning according to any one of claims 1 to 7 is implemented.

11. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the data processing method for cross-feature federated learning as described in any one of claims 1 to 7.

Images