Data processing method, device, equipment, storage medium and program product

Abstract

The present application provides a data processing method, apparatus, device, storage medium and program product, the method comprising: constructing a virtual feature correlation matrix based on first sample feature data and a pre-trained secure computing model; determining the collinearity quantization factor of each feature corresponding to the first sample feature data based on the feature correlation matrix; determining the target feature from each feature corresponding to the first sample feature data based on the collinearity quantization factor; deleting the feature data of the target feature from the first sample feature data to obtain the first training data for joint training; wherein the feature data of the target feature has a linear relationship with the feature data of at least one of the other features. In this way, under the premise of protecting data privacy, the data with collinearity in the feature data held by each participant can be screened and eliminated, which can improve the accuracy and stability of the federated model obtained by joint training and improve the modeling effect of the federated model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and is related to, but not limited to, a data processing method, apparatus, device, storage medium, and program product. Background Art

[0002] Machine learning is the study of how to use computers to simulate or implement human learning. It is one of the most intelligent and cutting-edge research fields in artificial intelligence. Machine learning research falls into two main directions: the first is traditional machine learning, which focuses on learning mechanisms and attempts to simulate human learning. The second is machine learning in big data environments, which focuses on how to effectively utilize information and extract hidden, effective, and understandable knowledge from massive amounts of data.

[0003] Federated learning is an emerging privacy-preserving technology that effectively combines data from multiple sources for model training without leaving local storage. Machine learning models can solve many business problems in the big data field. Eliminating collinear data is crucial for training effective models. While protecting data privacy, related technologies are unable to quantify the collinearity of multi-source data in federated learning, making it impossible to effectively filter and remove collinear training data. This results in low accuracy and poor stability of the trained models. Summary of the Invention

[0004] The embodiments of the present application provide a data processing method, apparatus, device, computer-readable storage medium, and computer program product, which can eliminate data with collinearity in linear federated modeling, improve the accuracy and stability of the federated model, and improve the modeling effect of the model.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] This embodiment of the present application provides a data processing method, which is applied to a first participant in federated learning. The method includes:

[0007] Constructing a virtual feature correlation matrix based on the first sample feature data held by the first party and a pre-trained secure computing model, where the secure computing model is pre-trained by the first party and other federated learning parties based on secure multi-party computing;

[0008] Determining, based on the feature correlation matrix, a collinearity quantization factor of each feature corresponding to the first sample feature data;

[0009] Determining a target feature from each feature corresponding to the first sample feature data based on the collinearity quantization factor;

[0010] Deleting the feature data of the target feature from the first sample feature data to obtain first training data for joint training between the first participant and the other participants;

[0011] There is a linear relationship between the feature data of the target feature and the feature data of at least one of the other features, and the other features include features held by the first participant except the target feature and features held by the other participants.

[0012] This embodiment of the present application provides a data processing device, applied to a first participant in federated learning, comprising:

[0013] a construction module, configured to construct a virtual feature correlation matrix based on the first sample feature data held by the first party and a pre-trained secure computing model, where the secure computing model is pre-trained by the first party and other federated learning parties based on secure multi-party computing;

[0014] A first determining module, configured to determine, based on the feature correlation matrix, a collinearity quantization factor of each feature corresponding to the first sample feature data;

[0015] A second determining module is configured to determine a target feature from each feature corresponding to the first sample feature data based on the collinearity quantization factor;

[0016] a deletion module, configured to delete the feature data of the target feature from the first sample feature data, and obtain first training data for joint training between the first participant and the other participants;

[0017] There is a linear relationship between the feature data of the target feature and the feature data of at least one of the other features, and the other features include features held by the first participant except the target feature and features held by the other participants.

[0018] An embodiment of the present application provides a data processing device, the device comprising:

[0019] a memory for storing executable instructions;

[0020] The processor is used to implement the method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0021] An embodiment of the present application provides a computer-readable storage medium having executable instructions stored thereon, which are used to cause a processor to execute and implement the method provided in the embodiment of the present application.

[0022] An embodiment of the present application provides a computer program product, including a computer program, which implements the method provided in the embodiment of the present application when executed by a processor.

[0023] The embodiments of the present application have the following beneficial effects:

[0024] In the data processing method provided in the embodiment of the present application, when performing data processing, first, the first participant and other participants of the federated learning pre-train based on secure multi-party computing to obtain a secure computing model; then the first participant obtains the first sample feature data held by the first participant, and constructs a virtual feature correlation matrix based on the first sample feature data and the pre-trained secure computing model; then, based on the feature correlation matrix, the collinearity quantization factor of each feature corresponding to the first sample feature data is determined; the target feature among each feature corresponding to the first sample feature data is determined by the collinearity quantization factor, and the feature data of these target features have a linear relationship with the feature data of at least one of the other features, and the other features here include not only the features held by the first participant except the target feature, but also all the features held by other participants; after determining the target feature, the feature data of the target feature is deleted from the first sample feature data to obtain the first training data for joint training by the first participant and other participants. In this way, under the premise of protecting data privacy, the data with collinearity in the feature data held by each participant can be screened and eliminated to obtain training data without linear relationship. Therefore, when conducting joint training, each participant can use the training data without linear relationship for joint training, which can improve the accuracy and stability of the federated model and improve the modeling effect of the federated model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of the network architecture of the data processing method provided in an embodiment of the present application;

[0026] Figure 2 Schematic diagram of the structure of the data processing device provided in the embodiment of the present application;

[0027] Figure 3 A schematic diagram of an implementation flow of the data processing method provided in an embodiment of the present application;

[0028] Figure 4 A schematic diagram of another implementation flow of the data processing method provided in an embodiment of the present application;

[0029] Figure 5 A schematic diagram of the calculation process of the variance inflation factor in the vertical federation scenario provided in an embodiment of the present application;

[0030] Figure 6 Schematic diagram of the calculation process of the determinant of the correlation matrix provided in the embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0032] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0033] In the following description, the terms "first\second\third" involved are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first\second\third" can be interchanged with the specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0035] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0036] 1) Vertical Federated Learning: When two datasets have a large overlap in users but a small overlap in user features, the datasets are split vertically (i.e., along the feature dimension) and the data with the same users but different user features is used for training.

[0037] 2) Variance Inflation Factor (VIF), also known as variance expansion factor, is a numerical value that characterizes the degree of multicollinearity between the observed values ​​of the independent variables. It is used to measure the severity of multicollinearity in the multiple linear regression model.

[0038] 3) Homomorphic Encryption: Homomorphic encryption is a cryptographic technique based on computational complexity theory, a mathematically challenging problem. Processing homomorphically encrypted data yields an output that, when decrypted, produces the same result as the original, unencrypted data processed using the same method.

[0039] The following describes exemplary applications of the apparatus for implementing the embodiments of the present application. The apparatus provided in the embodiments of the present application can be implemented as a terminal device. The following describes exemplary applications of the apparatus when implemented as a terminal device.

[0040] Figure 1 A schematic diagram of the network architecture of the data processing method provided in the embodiment of the present application is shown as follows: Figure 1 As shown, the network architecture includes at least a first participant 100, a second participant 200 and a network 300. To support an exemplary application, the first participant 100 and the second participant 200 can be participants in the joint training of machine learning models in vertical federated learning. The first participant 100 and the second participant 200 can be clients, such as participant devices of banks or hospitals that store user feature data. The client can be a laptop, tablet computer, desktop computer, dedicated training equipment, or other device with model training capabilities. The first participant 100 is connected to the second participant 200 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two, using wireless or wired links to achieve data transmission.

[0041] First party 100 first obtains first sample feature data from its own data, processes the first sample feature data, and obtains processed first sample feature data. Second party 200 obtains second sample feature data from its own data and processes the second sample feature data to obtain processed second sample feature data. The first sample feature data and the second sample feature data here have the same identifier, meaning that the first sample feature data and the second sample feature data represent different features of the same batch of samples held by first party 100 and second party 200, respectively. Based on secure multi-party computation, first party 100 and second party 200 use the processed first sample feature data and the processed second sample feature data to determine a first matrix E, which both parties possess. Based on secure multi-party computation, first party 100 can only obtain the first matrix E and cannot learn the processed second sample feature data held by second party 200. Similarly, second party 200 can only obtain the first matrix E and cannot learn the processed first sample feature data held by first party 100.

[0042] After obtaining the first matrix E, the first participant 100 constructs a virtual feature correlation matrix based on the first matrix E and the processed first sample feature data. The determinant of the feature correlation matrix and the determinants of the corresponding co-factors of the feature correlation matrix are then calculated. Based on the determinant of the feature correlation matrix and the determinants of the corresponding co-factors, a collinearity quantification factor is determined for each feature corresponding to the first sample feature data. This collinearity quantification factor can be a variance inflation factor, which quantifies the collinearity of each feature with all other features. This "all other features" includes not only the features held by the first participant 100, except for the currently quantified features, but also all features held by the second participant 200. Based on the collinearity quantification factor for each feature, the first participant 100 determines which features exhibit collinearity between their feature data and designates these features as target features. Finally, the target features' feature data is removed from the first sample feature data, resulting in first training data for joint training between the first participant 100 and the second participant 200. Using the same method, the second participant 200 obtains second training data for joint training. Therefore, when performing joint training, the first participant 100 and the second participant 200 use the first training data and the second training data without collinearity to perform joint training, and can obtain a federal model with high accuracy and good stability.

[0043] Through the method provided in the embodiment of the present application, it is possible to screen and eliminate data with collinearity in the feature data held by each participant while protecting data privacy, and obtain training data without linear relationships. Therefore, when conducting joint training, each participant can use the training data without linear relationships for joint training, which can improve the accuracy and stability of the federated model and improve the modeling effect of the federated model.

[0044] The apparatus provided in the embodiments of the present application may be implemented as hardware or a combination of hardware and software. Various exemplary implementations of the apparatus provided in the embodiments of the present application are described below.

[0045] according to Figure 2 The exemplary structure of the data processing device 10 is shown. Here, the data processing device 10 is shown as a device applied to the first participant of federated learning. Other exemplary structures of the data processing device 10 can be foreseen. Therefore, the structure described here should not be regarded as limiting. For example, some components described below can be omitted, or components not described below can be added to meet the special needs of certain applications.

[0046] Figure 2The data processing device 10 shown includes: at least one processor 110, a memory 140, at least one network interface 120, and a user interface 130. Each component in the data processing device 10 is coupled together via a bus system 150. It is understood that the bus system 150 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 150 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 150 is not described in detail. Figure 2 Various buses are labeled as bus system 150 .

[0047] The user interface 130 may include a display, a keyboard, a mouse, a touch pad, a touch screen, and the like.

[0048] The memory 140 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM). The volatile memory may be a random access memory (RAM). The memory 140 described in the embodiments of the present application is intended to include any suitable type of memory.

[0049] The memory 140 in the embodiments of the present application can store data to support the operation of the data processing device 10. Examples of such data include any computer program used to operate on the data processing device 10, such as an operating system and application programs. The operating system includes various system programs, such as a framework layer, a core library layer, and a driver layer, which are used to implement various basic services and handle hardware-based tasks. The application program can include various application programs.

[0050] As an example of the method provided in the embodiment of the present application being implemented by software, the method provided in the embodiment of the present application can be directly embodied as a combination of software modules executed by the processor 110. The software module can be located in a storage medium, and the storage medium is located in the memory 140. The processor 110 reads the executable instructions included in the software module in the memory 140, and combines with the necessary hardware (for example, including the processor 110 and other components connected to the bus 150) to complete the method provided in the embodiment of the present application.

[0051] As an example, the processor 110 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0052] The data processing method provided in the embodiments of the present application will be described in conjunction with the exemplary application and implementation of the device provided in the embodiments of the present application.

[0053] Figure 3 A schematic diagram of an implementation flow of the data processing method provided in the embodiment of the present application, which is applied to Figure 1 The first participant in the network architecture shown will combine Figure 3 The steps shown are explained.

[0054] Step S301: construct a virtual feature correlation matrix based on the first sample feature data held by the first participant and a pre-trained security calculation model.

[0055] In the embodiments of the present application, among the features corresponding to the first sample feature data, the feature data of different features may have a linear relationship. If a model is trained using data with collinearity, the trained model will have low accuracy and poor stability. To eliminate data with linear relationships, the first participant constructs a virtual feature correlation matrix, denoted as H, based on the first sample feature data and a pre-trained security calculation model.

[0056] The secure computing model is pre-trained by the first participant and other participants in the federated learning based on secure multi-party computing, and can perform matrix addition, subtraction, and matrix multiplication under privacy. Exemplarily, in the embodiment of the present application, the data processing method is described by taking two participants in federated learning as an example. In practical applications, the data processing method can also be applied to three or more participants in federated learning. Under the premise that the data held by each participant does not leave the local area and the data privacy is guaranteed, the first participant and the second participant perform training based on privacy protection technology to obtain a trained secure computing model, and the trained secure computing model is held by each participant.

[0057] In the embodiment of the present application, since the first participant and the second participant conduct joint training, the training data must come from the same user in order to conduct the joint training. That is, the first sample feature data used by the first participant for joint training and the second sample feature data used by the second participant for joint training have the same corresponding user ID (Identity Document). Therefore, before obtaining the first sample feature data, the first participant must first determine the common users held by each participant, and then filter out the target users participating in this training from these common users, and use the feature data of these target users as the first sample feature data.

[0058] Step S302: determining the collinearity quantization factor of each feature corresponding to the first sample feature data based on the feature correlation matrix.

[0059] In practical applications, methods for determining the quantitative factor of collinearity between the characteristic data of multiple features include the variance inflation factor method, eigenvalue analysis, and the condition number method. In addition, there is also an intuitive judgment method that can qualitatively analyze the degree of collinearity between the characteristic data of multiple features. However, this intuitive judgment method is generally used for preliminary judgment and cannot be used for quantitative analysis.

[0060] The first sample feature data held by the first participant and the second sample feature data held by the second participant correspond to data in different columns of the feature correlation matrix. Determining whether there is collinearity between the data in each column of the feature correlation matrix and the data in other columns can determine whether there is collinearity between the feature data of each feature in the first sample and the feature data of other features. Possible collinearity between feature data of different features includes the following: the feature data of one feature is a multiple of the feature data of another feature; the feature data of one feature is equal to the feature data of another feature plus a constant; or the feature data of one feature is equal to the sum of the feature data of two other features.

[0061] In an embodiment of the present application, the collinearity quantization factors of each feature corresponding to the first sample feature data are determined based on the feature correlation matrix. Based on the collinearity quantization factors of each feature corresponding to the first sample feature data, it can be determined which features have feature data that is collinear with it.

[0062] Step S303 : determining a target feature from the features corresponding to the first sample feature data based on the collinearity quantization factor.

[0063] According to the collinearity quantification factor, the target features with collinearity are determined. Taking the variance inflation factor as an example, in the absence of multicollinearity, the variance inflation factor is close to 1. The stronger the multicollinearity, the larger the variance inflation factor. In fact, there is always more or less multicollinearity between the data, so it is unrealistic to use the variance inflation factor equal to 1 as a standard for evaluating collinearity. Generally, a boundary value can be preset according to the actual application scenario. In the embodiment of the present application, the preset boundary value can be 10. Determine whether the variance inflation factor of each feature is greater than 10. When the variance inflation factor of a feature is greater than 10, it is considered that there is strong collinearity between the feature data of the feature and the feature data of other features, and the feature is determined as the target feature.

[0064] Step S304: Delete the feature data of the target feature from the first sample feature data to obtain first training data for joint training between the first participant and other participants.

[0065] There is a linear relationship between the feature data of the target feature and the feature data of at least one of the other features, where the other features include features other than the target feature held by the first participant and features held by other participants.

[0066] In this embodiment of the present application, the variance inflation factor of each feature in the first sample feature data of the first participant can be calculated based on the feature correlation matrix, and a target feature whose variance inflation factor is greater than a preset threshold value can be determined. The feature data of the target feature in the first sample feature data is then deleted to obtain feature data free of collinearity as the first training data. Thus, during joint training, the first participant can utilize the first training data free of collinearity for joint training, thereby obtaining a federated model with high accuracy and good stability.

[0067] The data processing method provided in the embodiment of the present application is applied to the first participant of federated learning. The data processing method includes: constructing a virtual feature correlation matrix based on the first sample feature data held by the first participant and a pre-trained secure computing model, wherein the secure computing model is pre-trained by the first participant and other participants of the federated learning based on secure multi-party computing; determining the collinearity quantization factor of each feature corresponding to the first sample feature data based on the feature correlation matrix; determining the target feature from each feature corresponding to the first sample feature data based on the collinearity quantization factor; deleting the feature data of the target feature from the first sample feature data to obtain first training data for joint training by the first participant and other participants; wherein the feature data of the target feature has a linear relationship with the feature data of at least one of the other features, and the other features include features other than the target feature held by the first participant and features held by other participants. Through this method, it is possible to screen and eliminate the data with collinearity in the feature data held by each participant under the premise of protecting data privacy, and obtain training data without linear relationship. Therefore, when conducting joint training, each participant uses the training data without linear relationship for joint training, which can improve the accuracy and stability of the federated model and improve the modeling effect of the federated model.

[0068] In some embodiments, the above Figure 3 In the illustrated embodiment, before step S301 , the first participant first obtains first sample feature data from the feature data held by itself.

[0069] Because the first and second participants are jointly training, the training data must come from the same users. This means that the first sample feature data used by the first participant for joint training and the second sample feature data used by the second participant for joint training have the same corresponding ID. To obtain the first sample feature data, it is first necessary to identify the common users held by each participant. Then, from these common users, the target users participating in this training are selected and their feature data is used as the first sample feature data.

[0070] The following example illustrates the process of the first participant obtaining the first sample feature data.

[0071] The first and second parties each hold a batch of feature data for different users. For example, the first party holds feature data for users with birthdays AA, ages BB, weights CC, and savings DD, while the second party holds feature data for users with ages BB, spending power EE, and hobbies FF. The database tables for the first and second parties are shown in Tables 1 and 2 below, respectively:

[0072] Table 1 Database table of the first participant

[0073] ID AA BB CC DD 01 a1 b1 c1 d1 02 a2 b2 c2 d2 03 a3 b3 c3 d3 04 a4 b4 c4 d4 06 a6 b6 c6 d6

[0074] Table 2 Database table of the second participant

[0075] ID BB EE FF 01 b1 e1 f1 03 b3 e3 f3 04 b4 e4 f4 05 b5 e5 f5 06 b6 e6 f6

[0076] According to the database table of the first participant, obtain the characteristic data held by the first participant According to the database table of the second party, obtain the characteristic data held by the second party

[0077] Under the premise of privacy protection, since each participant cannot know which users' data is stored by other participants, in the embodiment of the present application, the first participant and the second participant determine common users based on privacy protection technology. Determining common users based on privacy protection technology can be achieved as follows: the first participant and the second participant each obtain the identifier of the user they hold; then, based on privacy protection technology, the intersection of the identifier of the user held by the first participant and the identifier of the user held by the second participant is calculated, and the result is the common user. Using this method, it is determined that the common users held by the first participant and the second participant are users with IDs 01, 03, 04, and 06, denoted as S = {01, 03, 04, 06}.

[0078] After obtaining the common user S, the first participant determines the target users to participate in this training, such as randomly selecting random users from the common users as the target users to participate in this training, and then obtaining the feature data of these target users from its own stored data as the first sample feature data. In addition, the first participant sends the ID of the selected target users to the second participant, and the second participant obtains the feature data of these target users from its own stored data as the second sample feature data. Alternatively, the second participant may determine the target users to participate in this training and send the ID of the determined target user to the first participant. For example, the first participant randomly selects three users with IDs 01, 03, and 04 from S as target users, denoted as S1={01,03,04}. The first participant selects the feature data of the target user S1 from the feature data it holds and obtains the first sample feature data.

[0079] On the second participant side, the second participant filters the characteristic data of the target user S1 from the characteristic data held by itself to obtain the second sample characteristic data.

[0080] In the embodiment of the present application, through the above steps, under the premise that the data of each participant does not leave the local area and privacy protection is achieved, the first participant obtains the first sample feature data and the second participant obtains the second sample feature data.

[0081] In some embodiments, the above Figure 3 In the illustrated embodiment, step S301 of "building a virtual feature correlation matrix based on the first sample feature data held by the first participant and the pre-trained security computing model" can be implemented by the following steps:

[0082] Step S3011 : determining, based on the first sample feature data, feature data of each feature corresponding to the first sample feature data and the number of samples corresponding to the first sample feature data.

[0083] Still using the above example data for illustration, as shown in Table 3, the features corresponding to the first sample feature data are: AA, BB, CC and DD. The feature data of feature AA is {a1, a3, a4}, the feature data of feature BB is {b1, b3, b4}, the feature data of feature CC is {c1, c3, c4}, and the feature data of feature DD is {d1, d3, d4}. It can be seen that the feature data of each feature corresponds to a column of data in A.

[0084] The samples corresponding to the first sample feature data are: {a1, b1, c1, d1}, {a3, b3, c3, d3} and {a4, b4, c4, d4}, and the number of samples n is determined to be 3.

[0085] Step S3012: Calculate the mean and standard deviation of the feature data of each feature.

[0086] according to Calculate the mean of the feature data corresponding to features AA, BB, CC and DD, according to Calculate the standard deviation of the feature data corresponding to features AA, BB, CC, and DD, where x = a, b, c, d, and n = 3.

[0087] Step S3013, determining processed first sample feature data based on the feature data of each feature, the mean corresponding to the feature data of each feature, the standard deviation corresponding to the feature data of each feature, and the number of samples;

[0088] In the embodiment of the present application, the mean value corresponding to the characteristic data of each characteristic can be calculated. The standard deviation σ corresponding to the characteristic data of each feature x The feature data x of each feature is updated with the number of samples n to obtain the processed feature data x. The processing formula for each feature data x after processing can be Thus, the processed first sample feature data is obtained in, x=a,b,c,d,n=3.

[0089] Step S3014: input the processed first sample feature data into a security calculation model to obtain a first matrix.

[0090] In the embodiment of the present application, each participant is trained based on a secure multi-party computing protocol such as the SPDZ protocol or MASCOT to obtain a trained secure computing model. The trained model can perform matrix addition, subtraction and matrix multiplication under privacy. The research of secure multi-party computing (MPC) is mainly aimed at the problem of how to securely calculate an agreed function without a trusted third party. If there are m participants in the federated learning training, the trained secure computing model can be expressed as y 1,…,m =f(x 1,…,m ), where m≥2. After the training is completed, the m participants all hold the secure computing model, and each participant inputs its own private input value to obtain the corresponding output value. For example, the i-th participant inputs x i , the output is y i , where 1≤i≤m.

[0091] In the embodiment of this application, two participants are taken as an example, and the trained security calculation model is represented as y 1,2 =f(x 1,2), the first participant and the second participant both hold the model, and the first participant inputs the processed first sample feature data C into y 1,2 =f(x 1,2 ), get the first matrix E, the first matrix E=C T D, where D is the processed second sample feature data. It should be noted that the first matrix E=C T D is only used to illustrate that the value of E is equal to C T The value of D does not mean that E is based on C T D is calculated, the first party is based on y 1,2 =f(x 1,2 ) calculated, the first participant does not need to obtain the processed second sample feature data D from the second participant. T D shows that the first matrix E is a 4*3 matrix.

[0092] Correspondingly, on the second participant side, the second participant inputs the processed second sample feature data D into y 1,2 =f(x 1,2 ), get the first matrix E, the first matrix E=C T D, among which C T is the transposed matrix of the first sample feature data after processing. It should be noted that the first matrix E=C T D is only used to illustrate that the value of E is equal to C T The value of D does not mean that E is based on C T D is calculated, and the second party is based on y 1,2 =f(x 1,2 ) calculated, the second participant does not need to obtain the transposed matrix C of the processed first sample feature data from the first participant. T .

[0093] In the embodiment of the present application, the first matrix E is determined based on a secure multi-party computing protocol. Each participant cannot obtain the private data of other participants, thereby ensuring the privacy and security of the data of each participant.

[0094] Step S3015: construct a virtual feature correlation matrix based on the processed first sample feature data and the first matrix.

[0095] In the embodiment of the present application, “constructing a virtual feature correlation matrix” can be achieved by the following steps:

[0096] Step S30151: Determine a first symmetric matrix based on the processed first sample feature data.

[0097] Symmetric matrices refer to square matrices with the main diagonal as the axis of symmetry and the corresponding elements being equal. In the embodiment of the present application, the first symmetric matrix F is constructed based on the processed first sample feature data C, and the transposed matrix of the processed first sample data C can be used. And the first sample data after processing Determine the first symmetric matrix F, the calculation formula of F is F=C T The dimensions of C and F are 4*4.

[0098] Step S30152: Generate an empty matrix whose number of rows and columns are equal to the number of columns of the first matrix.

[0099] When constructing a feature correlation matrix based on the features held by the first participant and the features held by the second participant, under the premise that the data of each participant is not released locally, the first participant can only know the features held by the second participant, and cannot obtain the feature data of each feature held by the second participant. Therefore, in the embodiment of the present application, a virtual feature correlation matrix is ​​constructed on the first participant, in which the data related to the feature data of the second participant's features is empty.

[0100] According to step S3014 above, the first matrix E is a 4*3 matrix with 3 columns. Therefore, the generated empty matrix is ​​a 3*3 matrix, denoted as G'. The second symmetric matrix G corresponding to the empty matrix G' is determined by the second participant based on the second sample feature data B.

[0101] Step S30153: construct a virtual feature correlation matrix based on the first symmetric matrix, the first matrix, the transposed matrix of the first matrix, and the empty matrix.

[0102] According to the first symmetric matrix F, the first matrix E, the transposed matrix E of the first matrix T , and the empty matrix G', construct a virtual feature correlation matrix H, expressed as Among them, the dimension of the first symmetric matrix F is 4*4, the dimension of the first matrix E is 4*3, and the transposed matrix E of the first matrix is T The dimension of is 3*4, the dimension of the empty matrix G' is 3*3, so the dimension of the feature correlation matrix H is 7*7.

[0103] The method provided in the embodiment of the present application is that the first participant obtains a secure computing model through training with other participants based on a secure multi-party computing protocol. Under the premise that the data held by each participant does not leave the local area and the data privacy is guaranteed, the problem of collinearity of multi-party data in federated learning that cannot be quantified can be converted into a problem of block matrix determinant that can be handled. Therefore, the collinearity factor of multi-party data in federated learning can be quantified based on the feature correlation matrix, providing a basis for eliminating training data with collinearity and obtaining a federated model with accuracy and stability.

[0104] In some embodiments, the above Figure 3 In the illustrated embodiment, step S302 of “determining the collinearity quantization factor of each feature corresponding to the first sample feature data based on the feature correlation matrix” can be implemented by the following steps:

[0105] Step S3021, determining the determinant of the feature correlation matrix.

[0106] The determinant of the feature correlation matrix H is represented by |H|. In the embodiment of the present application, determining the determinant of the feature correlation matrix can be achieved through steps S30211 to S30214:

[0107] Step S30211: Generate a first random matrix whose determinant is a preset value and whose dimension is the same as that of the empty matrix.

[0108] When calculating the determinant of the feature correlation matrix, since the data of each participant cannot be exported locally, in order to ensure that private data is not leaked, a first random matrix is ​​generated in the embodiment of the present application to confuse the original data, thereby achieving privacy protection.

[0109] In this embodiment of the present application, the first random matrix generated by the first participant satisfies the following conditions: the determinant is a preset value and the dimensions are the same as those of the empty matrix G'. For ease of calculation, the preset value can be set to 1. According to step S30152 above, the empty matrix G' is a 3*3 matrix. Therefore, the generated first random matrix is ​​a 3*3 matrix with a determinant of 1, denoted as R1.

[0110] Step S30212: Input the feature correlation matrix and the first random matrix into the security calculation model to obtain a second matrix.

[0111] The secure computation model trained by the first and second participants is represented as y 1,2 =f(x 1,2 ), the first participant inputs the feature correlation matrix H and the first random matrix R1 into the model to obtain a second matrix J, the second matrix J = R1 (D-CA -1 B) R2, where A -1is the inverse of the first sample feature data A, D is the processed second sample feature data, R2 is the second random matrix generated by the second participant with a determinant of 1 and a dimension of 4*4 (the dimension is the same as the first symmetric matrix F). It should be noted that the second matrix J=R1(D-CA -1 B) R2 is only used to illustrate that the value of the second matrix J is equal to R1 (D-CA -1 B) The value of R2 does not mean that the second matrix J is based on R1 (D-CA -1 B) R2 is calculated, the first party is based on y 1,2 =f(x 1,2 ) calculated, the first participant does not need to obtain the processed second sample feature data D and the second random matrix R2 from the second participant.

[0112] In the embodiment of the present application, the determinants of R1 and R2 are both 1. When the determinant of R1 or R2 is not 1,

[0113] In an embodiment of the present application, the second matrix J is determined based on a secure multi-party computing protocol, and each participant uses the generated random matrix to obfuscate their own data, so that each participant cannot obtain the private data of other participants, thereby ensuring the privacy and security of each participant's data.

[0114] Step S30213: Calculate the determinants of the first symmetric matrix and the second matrix respectively.

[0115] According to step S30151, the first symmetric matrix F is obtained. According to step S30212, the second matrix J is obtained. The determinants of the first symmetric matrix F and the second matrix J are calculated respectively to obtain the determinant |F| of the first symmetric matrix and the determinant |J| of the second matrix.

[0116] Here, when the determinant |R1| of the generated first random matrix or the determinant |R1| of the second random matrix is ​​not 1, it is necessary to multiply the determinant |J| of the second matrix when calculating it. Right now |R1(D-CA -1 B)R2|.

[0117] Step S30214: multiply the determinant of the first symmetric matrix and the determinant of the second matrix to obtain the determinant of the characteristic correlation matrix.

[0118] The determinant |F| of the first symmetric matrix is ​​multiplied by the determinant |J| of the second matrix to obtain the determinant |H|=|F|*|J| of the characteristic correlation matrix.

[0119] In an embodiment of the present application, each participant calculates the determinant of a virtual feature correlation matrix based on a secure computing model, and the entire computing process does not require obtaining private data of other participants, thereby ensuring the data security of each participant.

[0120] Step S3022: Delete the data in the i-th row and i-th column of the characteristic correlation matrix to obtain the remainder formulas corresponding to the characteristic correlation matrix.

[0121] Where i = 1, 2, ..., m1, where m1 is the number of features corresponding to the first sample feature data. For the example above, the number of features of the first sample feature data A is 4, so m1 = 4. Since the first participant can only determine the collinearity quantization factor for each feature corresponding to the first sample feature data it holds, it only needs to obtain the first, second, third, and fourth co-minors corresponding to the feature correlation matrix.

[0122] Delete feature correlation matrix H 7*7 The data of the ith row and ith column of , and the remaining data do not change the original order, forming a matrix of dimension 6*6, which is the ith co-minor Hii corresponding to the ith feature of the first sample feature data. For example, when i is 2, the second co-minor H corresponding to feature BB is obtained. 22 .

[0123] Step S3023, determining the determinant of each co-formula.

[0124] In the embodiment of the present application, the calculation method for determining the determinant of each co-minor can be the same as the method for determining the determinant of the feature correlation matrix in step S3021. For example, determining the determinant of the co-minor corresponding to the i-th feature can be implemented as follows: generating a random matrix with a determinant of a preset value and the same dimension as the empty matrix; inputting the i-th co-minor and the random matrix into the security calculation model to obtain the matrix corresponding to the i-th co-minor; calculating the determinant of the first symmetric matrix and the matrix corresponding to the i-th co-minor respectively; multiplying the determinant of the first symmetric matrix with the determinant of the matrix matrix corresponding to the i-th co-minor to obtain the determinant |Hii| of the i-th co-minor.

[0125] Step S3024: Determine the collinearity quantization factor of each feature corresponding to the first sample feature data based on the determinant of the feature correlation matrix and the determinants of each co-factor.

[0126] In the embodiment of the present application, the variance inflation factor is used as an example to determine the collinearity quantization factor, and the variance inflation factor VIF of the i-th feature is i , can be determined based on the determinant |H| of the feature correlation matrix and the determinant |Hii| of the ith cofactor, and the calculation formula is shown in formula (1):

[0127]

[0128] Among them, when i=1, the obtained VIF1 is the collinear quantization factor of feature AA, when i=2, the obtained VIF2 is the collinear quantization factor of feature BB, when i=3, the obtained VIF3 is the collinear quantization factor of feature CC, and when i=4, the obtained VIF4 is the collinear quantization factor of feature DD.

[0129] According to the method provided in the embodiment of the present application, the first participant calculates the determinant of the feature correlation matrix and the determinant of the cofactor of each feature corresponding to the first sample feature data based on a secure multi-party computing protocol, thereby obtaining the variance inflation factor of each feature corresponding to the first sample feature data, providing a basis for eliminating training data with collinearity and obtaining a federated model with accuracy and stability.

[0130] In some embodiments, the above Figure 3 In the illustrated embodiment, step S303 of “determining a target feature from each feature corresponding to the first sample feature data based on the collinearity quantization factor” can be implemented by the following steps:

[0131] Step S3031 , determining whether the collinearity quantization factor of each feature corresponding to the first sample feature data is greater than a preset boundary value.

[0132] Here, the size of the collinearity quantization factor of each feature corresponding to the first sample feature data and the preset boundary value is determined, that is, the size of VIF1, VIF2, VIF3 and VIF4 and the preset boundary value is determined.

[0133] Step S3032: Determine the feature whose collinearity quantization factor is greater than a preset boundary value as the target feature.

[0134] Taking 10 as an example of the preset boundary value, if it is determined that VIF1 and VIF2 are greater than 10, feature AA corresponding to VIF1 and feature BB corresponding to VIF2 are determined as target features.

[0135] In the above example, the first participant's target feature AA is birthday, the first participant's target feature BB is age, and the second participant's feature BB is age. It is determined that there is a linear relationship between birthday and age (age equals current year minus birth year), and there is a linear relationship between age and age (equal, i.e., a multiple of 1). That is, there is a linear relationship between the feature data of the first participant's target feature AA, the feature data of the first participant's feature BB, and the feature data of the second participant's feature BB, and there is a linear relationship between the feature data of the first participant's target feature BB, the feature data of the first participant's feature AA, and the feature data of the second participant's feature BB.

[0136] After obtaining the target feature, execute step S304 to delete the feature data of the target feature in the first sample feature data, that is, delete the first sample feature data. The first training data is obtained by combining the first column data (feature data of feature AA) and the second column data (feature data of feature BB).

[0137] The same as the above step of determining the target feature among the features corresponding to the first sample feature data held by the first participant, the target feature among the features corresponding to the second sample feature data held by the second participant is determined to be the feature BB of the second participant, and the feature data of feature BB is deleted from the second sample feature data, that is, the second sample feature data is deleted. The first column of data (feature data of feature BB) is used to obtain the second training data

[0138] Then, the first participant and the second participant perform joint training based on the first training data A' and the second training data B'. The feature data with collinear relationship has been deleted from the first training data A' and the second training data B'. Therefore, when performing joint training, the first participant and the second participant use the first training data and the second training data without linear relationship to perform joint training. The obtained federal model has higher accuracy and better stability, which can improve the modeling effect of the federal model.

[0139] Based on the above embodiments, the present application further provides a data processing method. Figure 4 A schematic diagram of another implementation flow of the data processing method provided in the embodiment of the present application, which is applied to Figure 1 The network architecture shown in Figure 4 As shown, the data processing method includes the following steps:

[0140] In step S401, the first party and the second party determine a common user held by the first party and the second party based on a secure multi-party computing protocol.

[0141] Because the first and second participants are jointly training, the training data must come from the same users. This means that the first sample feature data used by the first participant for joint training and the second sample feature data used by the second participant for joint training have the same corresponding ID. To obtain the first sample feature data, it is first necessary to identify the common users held by each participant. Then, from these common users, the target users participating in this training are selected and their feature data is used as the first sample feature data.

[0142] Under the premise of privacy protection, since each participant cannot know which user data is stored by the other participants, in the embodiment of the present application, the first participant and the second participant determine a common user based on privacy protection technology. Determining a common user based on privacy protection technology can be achieved by: the first participant and the second participant each obtain the identifier of the user they hold; then, based on the privacy protection technology, calculate the intersection of the identifier of the user held by the first participant and the identifier of the user held by the second participant, and the result obtained is the common user.

[0143] Step S402: The first participant determines target users to participate in this training.

[0144] After obtaining the common users, the first participant determines the target users to participate in this training, such as randomly selecting random users from the common users as the target users to participate in this training.

[0145] Step S403: The first participant sends the identifier of the target user to the second participant.

[0146] In other embodiments, the second participant may determine the target users who will participate in the training. In this case, step S402 and step S403 may be replaced by:

[0147] Step S402': the second participant determines the target users to participate in this training.

[0148] Step S403': the second participant sends the identifier of the target user to the first participant.

[0149] Step S404: The first participant obtains the first sample feature data held by itself.

[0150] The first participant obtains the characteristic data of the target user from the data stored in the first participant as the first sample characteristic data.

[0151] In step S405 , the first participant constructs a virtual first feature correlation matrix based on the first sample feature data and a pre-trained security computing model.

[0152] In some embodiments, the first participant constructs the first feature correlation matrix as follows: determining the feature data of each feature corresponding to the first sample feature data and the number of samples corresponding to the first sample feature data based on the first sample feature data; respectively calculating the mean and standard deviation corresponding to the feature data of each feature; determining the processed first sample feature data based on the feature data of each feature, the mean corresponding to the feature data of each feature, the standard deviation corresponding to the feature data of each feature and the number of samples; inputting the processed first sample feature data into the security calculation model to obtain the first matrix; determining the first symmetric matrix based on the processed first sample feature data; generating a first empty matrix whose number of rows and columns are equal to the number of columns of the first matrix; and constructing a virtual first feature correlation matrix based on the first symmetric matrix, the first matrix, the transposed matrix of the first matrix and the first empty matrix.

[0153] For example, the first sample feature data is A; according to Process each feature data x in A to obtain the processed first sample feature data C; input the processed first sample feature data C into the pre-trained security calculation model y 1,…,m =f(x 1,…,m ), obtain the first matrix E; according to the transposed matrix C of the processed first sample feature data T and the processed first sample feature data C, determine the first symmetric matrix F = C T C; the first empty matrix generated by the first participant is G'; the first characteristic correlation matrix constructed by the first participant is obtained

[0154] Step S406: The first participant determines the collinearity quantization factor of each feature corresponding to the first sample feature data based on the first feature correlation matrix.

[0155] In some embodiments, the first participant determines the collinearity quantization factor of each feature corresponding to the first sample feature data, which can be implemented as follows: determining the determinant of the first feature correlation matrix; deleting the data in the i-th row and i-th column of the first feature correlation matrix to obtain each co-formula corresponding to the first feature correlation matrix, where i = 1, 2, ..., m1, m1 is the number of features corresponding to the first sample feature data; determining the determinant of each co-formula; based on the determinant of the first feature correlation matrix and the determinants of each co-formula, determining the collinearity quantization factor of each feature corresponding to the first sample feature data.

[0156] Delete the data in the i-th row and i-th column of the first feature correlation matrix H to obtain the i-th co-minor Hii corresponding to the i-th feature of the first sample feature data, calculate the determinant |H| of the first feature correlation matrix and the determinant |Hii| of the i-th co-minor, and determine the collinearity quantization factor of each feature corresponding to the first sample feature data (taking the variance inflation factor as an example)

[0157] Among them, determining the determinant |H| of the first characteristic correlation matrix can be implemented as follows: generating a first random matrix whose determinant is a preset value and whose dimension is the same as that of the first empty matrix; inputting the first characteristic correlation matrix and the first random matrix into the security calculation model to obtain a second matrix; calculating the determinants of the first symmetric matrix and the second matrix respectively; multiplying the determinant of the first symmetric matrix and the determinant of the second matrix to obtain the determinant of the first characteristic correlation matrix.

[0158] Similarly, determining the determinant |Hii| of the i-th co-minor corresponding to the i-th feature can be achieved as follows: generating a random matrix whose determinant is a preset value and whose dimension is the same as the first empty matrix; inputting the i-th co-minor and the random matrix into the secure computing model to obtain the matrix corresponding to the i-th co-minor; respectively calculating the determinants of the first symmetric matrix and the matrix corresponding to the i-th co-minor; multiplying the determinant of the first symmetric matrix by the determinant of the matrix matrix corresponding to the i-th co-minor to obtain the determinant |Hii| of the i-th co-minor.

[0159] In step S407 , the first participant determines a target feature from the features corresponding to the first sample feature data based on the collinearity quantization factor.

[0160] In some embodiments, the first participant determines the target feature based on the collinearity quantization factor, which can be implemented as follows: judging whether the collinearity quantization factor of each feature corresponding to the first sample feature data is greater than a preset boundary value; and determining the feature whose collinearity quantization factor is greater than the preset boundary value as the target feature.

[0161] Step S408: The first participant deletes the feature data of the target feature from the first sample feature data to obtain first training data for joint training between the first participant and other participants.

[0162] There is a linear relationship between the feature data of the target feature of the first participant and the feature data of at least one of the other features, and the other features include the features held by the first participant except the target feature and all the features held by the second participant.

[0163] Step S409: The second participant obtains the second sample feature data held by itself.

[0164] The second participant obtains the characteristic data of the target user from the data stored in the second participant as the second sample characteristic data.

[0165] In step S410 , the second participant constructs a virtual second feature correlation matrix based on the second sample feature data and a pre-trained security computing model.

[0166] In some embodiments, the second participant constructs the second feature correlation matrix as follows: determining the feature data of each feature corresponding to the second sample feature data and the number of samples corresponding to the second sample feature data based on the second sample feature data; respectively calculating the mean and standard deviation corresponding to the feature data of each feature; determining the processed second sample feature data based on the feature data of each feature, the mean corresponding to the feature data of each feature, the standard deviation corresponding to the feature data of each feature and the number of samples; inputting the processed second sample feature data into the security calculation model to obtain the first matrix; determining the second symmetric matrix based on the processed second sample feature data; generating a second empty matrix whose number of rows and columns are equal to the number of columns of the first matrix; and constructing a virtual first feature correlation matrix based on the second symmetric matrix, the first matrix, the transposed matrix of the first matrix, and the second empty matrix.

[0167] For example, the second sample feature data is B; according to Process each feature data x in B to obtain the processed second sample feature data D; input the processed second sample feature data D into the pre-trained security calculation model y 1,…,m =f(x 1,…,m ), obtain the first matrix E; according to the transposed matrix D of the processed second sample feature data T and the processed second sample feature data D, determine the second symmetric matrix G=D T D; the second empty matrix generated by the second participant is F'; the second characteristic correlation matrix constructed by the second participant is obtained

[0168] In step S411 , the second participant determines the collinearity quantization factor of each feature corresponding to the second sample feature data based on the feature correlation matrix.

[0169] In some embodiments, the second party determines the collinearity quantization factors of each feature corresponding to the second sample feature data, which can be implemented as follows: determining the determinant of the second feature correlation matrix; deleting the data in the i+m1th row and i+m1th column of the second feature correlation matrix (because the first m1 columns of the second feature correlation matrix H' are the features of the first sample feature data, and therefore m1 needs to be added when determining the collinearity quantization factors of each feature corresponding to the second sample feature data), and obtaining each co-formula corresponding to the second feature correlation matrix, where i=1, 2,…, m2, m2 is the number of features corresponding to the second sample feature data; determining the determinant of each co-formula; and determining the collinearity quantization factors of each feature corresponding to the second sample feature data based on the determinant of the second feature correlation matrix and the determinants of each co-formula.

[0170] Delete the data of the i+m1th row and i+m1th column of the second feature correlation matrix H' to obtain the i-th co-formula H' corresponding to the i-th feature of the second sample feature data i+m1i+m1 , calculate the determinant of the second characteristic correlation matrix |H'| and the determinant of the i-th cofactor |H' i+m1i+m1 |, determine the collinearity quantization factor of each feature corresponding to the first sample feature data (taking the variance inflation factor as an example)

[0171] Among them, determining the determinant |H'| of the second characteristic correlation matrix can be implemented as follows: generating a second random matrix whose determinant is a preset value and whose dimension is the same as that of the second empty matrix; inputting the second characteristic correlation matrix and the second random matrix into the security calculation model to obtain the second matrix; calculating the determinants of the second symmetric matrix and the second matrix respectively; multiplying the determinant of the second symmetric matrix and the determinant of the second matrix to obtain the determinant of the second characteristic correlation matrix.

[0172] Similarly, determine the determinant of the ith cofactor corresponding to the ith feature |H' i+m1i+m1 |, which can be implemented as follows: generate a random matrix with a preset determinant and the same dimension as the second empty matrix; input the i-th co-minor and the random matrix into the secure calculation model to obtain the matrix corresponding to the i-th co-minor; calculate the determinants of the second symmetric matrix and the matrix corresponding to the i-th co-minor respectively; multiply the determinant of the second symmetric matrix by the determinant of the matrix matrix corresponding to the i-th co-minor to obtain the determinant of the i-th co-minor |H' i+m1i+m1 |.

[0173] In step S412, the second participant determines a target feature from the features corresponding to the second sample feature data based on the collinearity quantization factor.

[0174] In some embodiments, the second party determines the target feature based on the collinearity quantization factor, which can be implemented as follows: judging whether the collinearity quantization factor of each feature corresponding to the second sample feature data is greater than a preset boundary value; and determining the feature whose collinearity quantization factor is greater than the preset boundary value as the target feature.

[0175] In step S413, the second participant deletes the feature data of the target feature from the second sample feature data to obtain second training data for joint training between the second participant and other participants.

[0176] There is a linear relationship between the feature data of the target feature in the second participant and the feature data of at least one of the other features, and the other features include the features held by the second participant except the target feature and all the features held by the first participant.

[0177] In other embodiments, after step S413, the first party and the second party use the first training data and the second training data to conduct a joint model based on a secure multi-party computing protocol. Since there is no training data with a linear relationship between the first training data and the second training data, the accuracy and stability of the trained federated model can be improved, thereby improving the modeling effect of the federated model.

[0178] The data processing method provided by the embodiment of the present application is that the first participant and the second participant pre-train based on secure multi-party computing to obtain a secure computing model; then the first participant and the second participant respectively obtain first sample feature data and second sample feature data, the first participant constructs a virtual first feature correlation matrix based on the first sample feature data and the secure computing model, and the second participant constructs a virtual second feature correlation matrix based on the second sample feature data and the secure computing model; then the first participant determines the collinearity quantization factor of each feature corresponding to the first sample feature data based on the first feature correlation matrix, and the second participant determines the collinearity quantization factor of each feature corresponding to the second sample feature data based on the second feature correlation matrix; the first participant obtains the collinearity quantization factor of each feature corresponding to the second sample feature data based on the second feature correlation matrix; The collinearity quantization factor of each feature corresponding to the first sample feature data determines the target feature among the features corresponding to the first sample feature data. The second participant determines the target feature among the features corresponding to the second sample feature data based on the collinearity quantization factor of each feature corresponding to the second sample feature data. The feature data of these target features have a linear relationship with the feature data of at least one of the other features. Here, the other features include all features held by the first and second participants except the target feature. After determining the target feature, the first participant deletes the feature data of the target feature from the first sample feature data to obtain first training data. The second participant deletes the feature data of the target feature from the second sample feature data to obtain second training data. In this way, under the premise of protecting data privacy, the data with collinearity in the feature data held by the first and second participants can be screened and eliminated to obtain training data without linear relationship. Therefore, when conducting joint training, the first and second participants use the training data without linear relationship for joint training, which can improve the accuracy and stability of the federated model and improve the modeling effect of the federated model.

[0179] The following describes an exemplary application of the embodiments of the present application in a practical application scenario.

[0180] Vertical federated learning typically involves different participants jointly training machine learning models. During the linear modeling process, collinearity can significantly impact the model's stability and performance, and therefore needs to be eliminated. The variance inflation factor (VIF) effectively quantifies the collinearity between a single feature and all other features and is very commonly used in practical modeling. In related technologies, calculating VIF typically requires aggregating data from all parties in a single location. However, data from various parties (such as banks and enterprises) may involve personal privacy or commercial secrets, and directly making it available to other participants can result in information leakage. Feature screening solutions for linear federated modeling in related technologies primarily target single-column data and lack methods like VIF that can characterize collinearity across multiple columns. Related technologies are unable to calculate VIF by combining data from multiple parties while protecting data privacy.

[0181] The embodiment of the present application is aimed at the scenario of two independent participants, and it can be assumed that data joint modeling is carried out between two companies. Due to compliance, privacy, and commercial confidentiality reasons, the original data of each company cannot leave their respective companies. The intermediate data exchanged during the modeling process cannot derive or leak unnecessary original data information. The two companies each hold different feature data of multiple users with the same ID. There may be collinearity between these features, which may affect the modeling effect of the subsequent linear model. The embodiment of the present application uses the privacy and security technical solutions of the two parties to calculate the variance inflation factor VIF, and performs feature screening based on this.

[0182] Figure 5 A schematic diagram of the calculation process of the variance inflation factor in the vertical federation scenario provided in an embodiment of the present application is provided. Figure 6 The calculation flow diagram of the determinant of the correlation matrix provided in the embodiment of the present application is as follows: Figure 5 and Figure 6 , a detailed description is given of the calculation method of the variance inflation factor in the vertical federation scenario provided in the embodiment of the present application.

[0183] 1) In an embodiment of the present application, two independent parties (respectively referred to as Alice and Bob) each hold data of different characteristics of the same batch of IDs, respectively referred to as matrices A and B (where A and B have the same number of rows n, A has the same number of columns m1, and B has the same number of columns m2).

[0184] 2) Alice and Bob locally normalize the features (columns of matrices) of A and B respectively to obtain matrices C and D. The normalization method is: x represents a column of features, represents the mean of the feature, σ x Represents the standard deviation of the feature.

[0185] 3) Through the extended SPDZ protocol, perform matrix addition and subtraction and matrix multiplication under privacy and calculate the matrix product C T D=E, the calculation result is held by both Alice and Bob.

[0186] In the embodiment of the present application, the secure multi-party computing protocol SPDZ is used to implement matrix addition, subtraction and matrix multiplication calculations under privacy protection. Each participant inputs his or her own private data to obtain the calculation results, and cannot obtain the private data of other participants. This ensures that the entire calculation process does not leak any additional information other than the results, and has both high security and practical computing efficiency.

[0187] 4) Alice locally calculates the matrix multiplication C T C = F, Bob locally calculates the matrix multiplication D T D=G.

[0188] 5) The previous step obtained the Pearson correlation moment

[0189] Note 1: Here Alice holds F, E, E T , Bob holds E, E T 、G.

[0190] According to the VIF definition, the i-th feature Here, Hii ​​is the cofactor of the matrix H (i.e., the matrix remaining after deleting the i-th row and i-th column of the matrix), and |H| represents the determinant of the matrix H.

[0191] 6) Assuming that the i-th feature is on Alice's side (on Bob's side, we need the index i plus the number of columns m1 of Fii), then Alice can delete the i-th row and i-th column of matrix F to get matrix Fii. Both Alice and Bob can delete the i-th row of matrix E to get E i* , delete the matrix E T The i-th column of Then the remainder

[0192] Note 2: Here Alice holds Fii, E i* 、 Bob holds E i* 、

[0193] In the embodiment of the present application, the calculation problem of the determinant of the original matrix H, which cannot be processed, is converted into a processable block matrix Fii, E i* 、 and The problem of calculating the determinant of .

[0194] The following solution is based on the block matrix Fii, E i* 、 and The problem of calculating the determinant of .

[0195] 7) Based on Note 1 and Note 2, calculate |H ii | and |H| need to solve the same problem. When calculating |H|, the following M1, M2, M3, and M4 correspond to F, E, and E respectively. T , G, in the calculation |H ii |, the following M1, M2, M3, M4 correspond to Fii, E i* 、 .

[0196] Alice holds matrices M1, M2, and M3, and Bob holds matrices M2, M3, and M4. Calculate the determinant

[0197] 8) From here, we can solve the problem in 7) (Since Alice cannot obtain M4 and Bob cannot obtain M1, the determinant in 7) cannot be directly calculated). When M1 is reversible,

[0198] 9) Alice locally computes |M1| and And locally generate a random matrix R1 with determinant 1 and the same dimension as M4. Bob locally generates a random matrix R2 with determinant 1 and the same dimension as M4.

[0199] 10) Again use the extended SPDZ protocol to calculate the matrix subtraction and multiplication and recover the result matrix J = R1

[0200] Due to the existence of the random matrix, neither Alice nor Bob can recover the other's original matrix through J.

[0201] In this embodiment, a special random matrix is ​​constructed to obfuscate the original matrix, transforming the determinant calculation problem of the matrix involving the original data information into the determinant calculation problem of a random matrix with the same determinant. By obfuscating the original matrix with the random matrix, each participant cannot recover the data of another participant, thus protecting the privacy of each participant's data and preventing data leakage.

[0202] 11) Alice's local calculation Thus we get the determinant Solved the problem in 7).

[0203] 12) According to the methods described in 7) to 11), calculate |H11|, |H22|, ..., and |H|, thus obtaining VIF1, VIF2, …, VIF m1+m2 .

[0204] The method provided in the embodiment of the present application converts the original determinant calculation problem that cannot be handled into a block matrix determinant problem that can be handled through a clever matrix transformation; by constructing a special random matrix to confuse the original matrix, the determinant problem of the matrix involving the original data information is converted into the determinant calculation problem of the random matrix with the same determinant; through the clever cooperation of the SPDZ protocol, the entire calculation process does not leak any additional information other than the result, and has both high security and practical computing efficiency. The method provided in the embodiment of the present application makes it possible to safely calculate the variance inflation factor VIF, thereby enabling efficient feature screening and improving the overall effect of the subsequent linear model; and taking into account both high security and practicality, no unnecessary data information is leaked except for the result, and the overall computing overhead is controlled within the efficiency range of production practicality.

[0205] The following continues to describe an exemplary structure of a data processing device provided in an embodiment of the present application implemented as a software module. In some embodiments, such as Figure 2 As shown, the data processing device 70 stored in the memory 140 is applied to the second participant who performs joint training on the model. The software modules in the data processing device 70 may include:

[0206] A construction module 71 is configured to construct a virtual feature correlation matrix based on the first sample feature data held by the first party and a pre-trained secure computing model, where the secure computing model is pre-trained by the first party and other federated learning parties based on secure multi-party computing.

[0207] A first determining module 72 is configured to determine, based on the feature correlation matrix, a collinearity quantization factor of each feature corresponding to the first sample feature data;

[0208] A second determining module 73 is configured to determine a target feature from the features corresponding to the first sample feature data based on the collinearity quantization factor;

[0209] A deletion module 74 is configured to delete the feature data of the target feature from the first sample feature data to obtain first training data for joint training between the first participant and the other participants;

[0210] There is a linear relationship between the feature data of the target feature and the feature data of at least one of the other features, and the other features include features held by the first participant except the target feature and features held by the other participants.

[0211] In some embodiments, the building block 71 further includes:

[0212] a first determining submodule, configured to determine, based on the first sample characteristic data, characteristic data of each characteristic corresponding to the first sample characteristic data and the number of samples corresponding to the first sample characteristic data;

[0213] A calculation submodule, used to calculate the mean and standard deviation corresponding to the feature data of each feature;

[0214] a second determining submodule, configured to determine processed first sample feature data based on the feature data of each feature, the mean corresponding to the feature data of each feature, the standard deviation corresponding to the feature data of each feature, and the number of samples;

[0215] An input submodule, configured to input the processed first sample feature data into the security calculation model to obtain a first matrix;

[0216] A construction submodule is used to construct a virtual feature correlation matrix based on the processed first sample feature data and the first matrix.

[0217] In some embodiments, the building block further comprises:

[0218] a determining unit, configured to determine a first symmetric matrix according to the processed first sample feature data;

[0219] A first generating unit is used to generate an empty matrix with the number of rows and the number of columns both being equal to the number of columns of the first matrix;

[0220] A construction unit is used to construct a virtual characteristic correlation matrix according to the first symmetric matrix, the first matrix, the transposed matrix of the first matrix and the empty matrix.

[0221] In some embodiments, the first determining module 72 further includes:

[0222] A third determining submodule is used to determine the determinant of the feature correlation matrix;

[0223] a deletion submodule, configured to delete the data in the i-th row and i-th column of the feature correlation matrix to obtain the remainders corresponding to the feature correlation matrix, where i=1, 2, ..., m1, and m1 is the number of features corresponding to the first sample feature data;

[0224] a fourth determining submodule, configured to determine the determinants of the co-minors;

[0225] The fifth determining submodule is configured to determine, based on the determinant of the feature correlation matrix and the determinants of the co-minors, a collinearity quantization factor of each feature corresponding to the first sample feature data.

[0226] In some embodiments, the third determining submodule further includes:

[0227] A second generating unit is configured to generate a first random matrix having a determinant of a preset value and the same dimension as the empty matrix;

[0228] an input unit, configured to input the feature correlation matrix and the first random matrix into the security calculation model to obtain a second matrix;

[0229] a first calculation unit, configured to calculate the determinants of the first symmetric matrix and the second matrix respectively;

[0230] The second calculation unit is used to multiply the determinant of the first symmetric matrix and the determinant of the second matrix to obtain the determinant of the feature correlation matrix.

[0231] In some embodiments, the second determining module 73 further includes:

[0232] a judgment submodule, configured to judge whether a collinearity quantization factor of each feature corresponding to the first sample feature data is greater than a preset boundary value;

[0233] The sixth determining submodule is configured to determine a feature whose collinearity quantization factor is greater than a preset boundary value as a target feature.

[0234] It should be noted that the description of the above data processing device embodiment is similar to the description of the above method embodiment and has the same beneficial effects as the method embodiment. For technical details not disclosed in the data processing device embodiment of this application, those skilled in the art should refer to the description of the method embodiment of this application for understanding.

[0235] The present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data processing method described above in the present invention.

[0236] The embodiment of the present application provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the method provided by the embodiment of the present application, for example, Figures 3 to 6 The method shown.

[0237] In some embodiments, the storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EE PROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.

[0238] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0239] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (for example, files storing one or more modules, subroutines, or code portions).

[0240] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0241] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: Applied to a first participant in federated learning, the method includes: Constructing a virtual feature correlation matrix based on the first sample feature data held by the first participant and a pre-trained secure computing model, wherein the secure computing model is pre-trained by the first participant and other participants in federated learning based on secure multi-party computing. The step of constructing a virtual feature correlation matrix based on the first sample feature data held by the first participant and a pre-trained security computing model includes: determining, based on the first sample feature data, feature data of each feature corresponding to the first sample feature data and the number of samples corresponding to the first sample feature data; Calculate the mean and standard deviation of the feature data corresponding to each feature respectively; Determining processed first sample feature data based on the feature data of each feature, the mean corresponding to the feature data of each feature, the standard deviation corresponding to the feature data of each feature, and the number of samples; Inputting the processed first sample feature data into the secure computing model to obtain a first matrix, wherein the first matrix is ​​determined based on a secure multi-party computing protocol; constructing a virtual feature correlation matrix based on the processed first sample feature data and the first matrix; Determining, based on the feature correlation matrix, a collinearity quantization factor of each feature corresponding to the first sample feature data; Determining a target feature from each feature corresponding to the first sample feature data based on the collinearity quantization factor; Deleting the feature data of the target feature from the first sample feature data to obtain first training data for joint training between the first participant and the other participants; wherein the feature data of the target feature has a linear relationship with the feature data of at least one of the other features, and the other features include features held by the first participant other than the target feature and features held by the other participants; The determining, based on the feature correlation matrix, of the collinearity quantization factor of each feature corresponding to the first sample feature data includes: determining a determinant of the feature correlation matrix; Deleting the data in the i-th row and i-th column of the feature correlation matrix to obtain the co-formulas corresponding to the feature correlation matrix, where i=1, 2, ..., m 1 , m 1 is the number of features corresponding to the first sample feature data; Determining the determinants of the co-factors; Based on the determinant of the feature correlation matrix and the determinants of the co-factors, a collinearity quantization factor of each feature corresponding to the first sample feature data is determined.

2. The method according to claim 1, characterized in that The constructing a virtual feature correlation matrix according to the processed first sample feature data and the first matrix includes: determining a first symmetric matrix according to the processed first sample feature data; Generate an empty matrix with the number of rows and columns equal to the number of columns of the first matrix; A virtual feature correlation matrix is ​​constructed according to the first symmetric matrix, the first matrix, a transposed matrix of the first matrix, and the empty matrix.

3. The method according to claim 1, characterized in that Determining the determinant of the feature correlation matrix includes: Generate a first random matrix with a determinant of a preset value and the same dimension as the empty matrix; Inputting the feature correlation matrix and the first random matrix into the security calculation model to obtain a second matrix; Calculating the determinants of the first symmetric matrix and the second matrix respectively; The determinant of the feature correlation matrix is ​​obtained by multiplying the determinant of the first symmetric matrix and the determinant of the second matrix.

4. The method according to claim 1, wherein The determining, based on the collinearity quantization factor, a target feature from each feature corresponding to the first sample feature data includes: Determining whether a collinearity quantization factor of each feature corresponding to the first sample feature data is greater than a preset boundary value; The features whose collinearity quantization factors are greater than the preset boundary value are determined as target features.

5. A data processing device, characterized in that: Applied to a first participant in federated learning, the apparatus includes: A construction module is used to construct a virtual feature correlation matrix based on the first sample feature data held by the first participant and a pre-trained secure computing model, wherein the secure computing model is pre-trained by the first participant and other participants in federated learning based on secure multi-party computing, wherein the construction module is used to determine the feature data of each feature corresponding to the first sample feature data and the number of samples corresponding to the first sample feature data based on the first sample feature data; respectively calculate the mean and standard deviation corresponding to the feature data of each feature; determine the processed first sample feature data based on the feature data of each feature, the mean corresponding to the feature data of each feature, the standard deviation corresponding to the feature data of each feature, and the number of samples; input the processed first sample feature data into the secure computing model to obtain a first matrix, wherein the first matrix is ​​determined based on a secure multi-party computing protocol; and construct a virtual feature correlation matrix based on the processed first sample feature data and the first matrix; a first determining module, configured to determine, based on the feature correlation matrix, a collinearity quantization factor of each feature corresponding to the first sample feature data, wherein the first determining module is specifically configured to determine a determinant of the feature correlation matrix; delete data in the i-th row and i-th column of the feature correlation matrix to obtain co-minors corresponding to the feature correlation matrix, wherein i=1, 2, ..., m 1 , m 1 being the number of features corresponding to the first sample feature data; determine the determinants of the co-minors; and determine the collinearity quantization factor of each feature corresponding to the first sample feature data based on the determinant of the feature correlation matrix and the determinants of the co-minors; A second determining module is configured to determine a target feature from each feature corresponding to the first sample feature data based on the collinearity quantization factor; A deletion module is used to delete the feature data of the target feature from the first sample feature data to obtain first training data for joint training between the first participant and the other participants.

6. A data processing device, characterized in that: The device comprises: a memory for storing executable instructions; A processor, configured to implement the method according to any one of claims 1 to 4 when executing the executable instructions stored in the memory.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores executable instructions for causing a processor to execute the instructions to implement the method according to any one of claims 1 to 4.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images