Data multi-party joint processing method and system, server and terminal device

Abstract

The present disclosure provides a data multi-party joint processing method, system, server and terminal device. The method comprises: generating an encryption key; sending the encryption key to a data owner, the data owner encrypting its own original data using the encryption key to obtain encrypted data; receiving the encrypted data sent by the data owner and performing aggregation processing on the encrypted data to obtain a relationship matrix; obtaining a first intermediate matrix and a first perturbation matrix according to the relationship matrix, the data owner obtaining a second perturbation matrix according to the first perturbation matrix and its own original data; receiving the second perturbation matrix sent by the data owner and obtaining a target diagonal matrix according to the second perturbation matrix, and sending the target diagonal matrix to each data owner, so that the data owner obtains a distribution verification value of its own original data according to the target diagonal matrix and the first intermediate matrix. Thus, cross-institutional sharing and fusion of data are realized, and joint modeling of a mixed linear model is completed under the premise of ensuring data privacy and security.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a method, system, server, and terminal device for multi-party data processing. Background Technology

[0002] Data used to train machine learning models is often owned by different organizations and institutions, especially data with highly sensitive privacy. Due to concerns about data security, user privacy, and legal regulations, data owners are reluctant to directly exchange raw data, hindering effective data aggregation and impacting machine learning performance, thus limiting the improvement of machine learning models. In the field of genomics, the amount of genomic data held by various parties is even more limited. To promote better data mining, large-scale sample aggregation and joint analysis are needed. How to collaboratively train machine learning models among multiple data owners while ensuring data integrity has become a major challenge in data sharing and unlocking data value. Summary of the Invention

[0003] This disclosure aims to at least partially address one of the technical problems in the related art.

[0004] Therefore, one objective of this disclosure is to propose a multi-party collaborative data processing method to achieve cross-institutional data sharing and fusion, and complete MLM modeling, while ensuring data privacy and security.

[0005] The second objective of this disclosure is to propose a second method for multi-party collaborative data processing.

[0006] The third objective of this disclosure is to propose a server.

[0007] The fourth objective of this disclosure is to provide a terminal device.

[0008] The fifth objective of this disclosure is to propose a multi-party collaborative data processing system.

[0009] To achieve the above objectives, a first aspect of this disclosure proposes a multi-party collaborative data processing method, wherein the original data is distributed among K data owners, where K is an integer greater than or equal to 2, and the method includes:

[0010] Generate an encryption key; send the encryption key to at least one of the data owners, so that at least one of the data owners can use the encryption key to encrypt its own original data to obtain encrypted data; receive encrypted data sent by at least two of the data owners, and aggregate the encrypted data of at least two of the data owners to obtain a relation matrix; obtain a first intermediate matrix and a first perturbation matrix according to the relation matrix, and send the first intermediate matrix and the first perturbation matrix to the K data owners, so that the K data owners can obtain a second perturbation matrix according to the first perturbation matrix and their own original data; receive the second perturbation matrix sent by the K data owners, and obtain a target diagonal matrix according to the second perturbation matrix, and send the target diagonal matrix to each of the data owners, so that the K data owners can obtain the distribution test value of their own original data according to the target diagonal matrix and the first intermediate matrix.

[0011] The data multi-party joint processing method of this disclosure embodiment generates an encryption key and sends it to the data owner. The data owner uses the encryption key to encrypt its own original data, obtains encrypted data, and returns the encrypted data. After obtaining multiple returned encrypted data, the encrypted data is aggregated to obtain a corresponding relation matrix. Then, a first intermediate matrix and a first perturbation matrix are obtained based on the relation matrix, and the first intermediate matrix and the first perturbation matrix are sent to the corresponding data owner, so that the data owner obtains a second perturbation matrix based on the first perturbation matrix and its own original data. The target diagonal matrix is ​​obtained through the received second perturbation matrix and sent to the data owner. The data owner obtains the distribution test value of its own original data based on the target diagonal matrix and the first intermediate matrix. Thus, under the premise of ensuring data privacy and security, cross-institutional sharing and data fusion are realized, and MLM modeling is completed.

[0012] To achieve the above objectives, a second aspect of this disclosure proposes a second method for multi-party joint data processing, wherein the original data is distributed among K data owners, where K is an integer greater than or equal to 2, and the method includes the following steps:

[0013] The system receives a first intermediate matrix and a first perturbation matrix sent by the server, and obtains a second perturbation matrix based on the first perturbation matrix and its own original data; it sends the second perturbation matrix to the server so that the server obtains a target diagonal matrix based on the second perturbation matrix; it receives the target diagonal matrix sent by the server, and obtains the distribution test value of its own original data based on the target diagonal matrix and the first intermediate matrix.

[0014] Another data multi-party joint processing method according to an embodiment of this disclosure involves receiving a first intermediate matrix and a first perturbation matrix sent by a server, and obtaining a second perturbation matrix based on the first perturbation matrix and its own original data; sending the obtained second perturbation matrix to the server so that the server obtains a target diagonal matrix based on the second perturbation matrix; and receiving the target diagonal matrix sent by the server and obtaining the distribution test value of its own original data based on the target diagonal matrix. This method achieves cross-institutional data sharing and fusion while ensuring data privacy and security, and completes MLM modeling.

[0015] To achieve the above objectives, a third aspect of this disclosure provides a server comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the first data multi-party joint processing method as described in the first aspect of the above embodiment.

[0016] To achieve the above objectives, a fourth aspect of this disclosure provides a terminal device including a memory and a processor. The memory stores a computer program, and the processor executes the program to implement the second data multi-party joint processing method as described in the second aspect of the above embodiment.

[0017] To achieve the above objectives, a fifth aspect of this disclosure provides a data multi-party joint processing system, which includes a server as described in the third aspect of this disclosure and a terminal device as described in the fourth aspect of this disclosure, wherein K is an integer greater than or equal to 2.

[0018] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0019] Figure 1 This is a flowchart of the data multi-party joint processing method according to the first embodiment of this disclosure;

[0020] Figure 2 This is a flowchart of a method for generating a first random orthogonal matrix according to an embodiment of this disclosure;

[0021] Figure 3 This is a flowchart of a method for standardizing raw data according to an embodiment of this disclosure;

[0022] Figure 4 This is a flowchart of a data multi-party joint processing method according to the second embodiment of this disclosure;

[0023] Figure 5 This is a structural block diagram of a server according to an embodiment of the present disclosure. Detailed Implementation

[0024] Embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0025] The following description, with reference to the accompanying drawings, outlines a data multi-party collaborative processing method, system, server, and terminal device according to embodiments of this disclosure.

[0026] First, it should be noted that the data multi-party joint processing method of this disclosure involves multiple data owners and one or more servers. The original data is distributed among K data owners, where K is an integer greater than or equal to 2.

[0027] The raw data can be gene locus data, protein structure data, etc., and the data owner can be a company, research institution, etc. that owns the above data. The data owner will store the above raw data in its database.

[0028] In the field of genomics, the genomic data held by various parties is becoming increasingly limited. To promote better data mining, large amounts of samples need to be aggregated and analyzed jointly. How to jointly train machine learning models by multiple data owners while ensuring that the original data is not leaked has become a major challenge in data openness, sharing, and unlocking the value of data.

[0029] One approach is to apply the MLM (Mixed Linear Model) method to GWAS (Genome-wide Association Study). GWAS is an experimental design widely used to detect associations between genetic variations and phenotypes. By comparing the genetic information characteristics of a large population, it identifies specific genetic variations and uses statistical methods to find genetic factors associated with complex diseases, revealing genes and regulatory mechanisms related to disease occurrence and development. MLM has advantages such as better correction for population stratification and individual associations.

[0030] However, MLM methods generally require uniform standardization and variance estimation extraction for all data. Therefore, when multiple parties share some training data and wish to jointly build an MLM model, how to achieve variance estimation and parameter fitting of the MLM model without disclosing private data becomes an important issue.

[0031] Figure 1This is a flowchart of a data multi-party collaborative processing method according to the first embodiment of this disclosure. The executing entity in this embodiment can be a server, which may include a generation server and a decomposition server. This server may be owned by a third party, or a portion of the server may be owned by a third party, while another portion may be owned by one or more data owners, wherein the third party may not be any data owner. Figure 1 As shown, the method includes the following steps:

[0032] Step S110: Generate an encryption key.

[0033] The encryption key is generated by the aforementioned server and is used to encrypt the original data, ensuring its privacy and security during transmission. In some implementations, the encryption key includes a first random orthogonal matrix and a first random seed.

[0034] In this embodiment, the first random orthogonal matrix is ​​represented by the letter Q. The generation server first needs to obtain the sample count of all data owners to get the total sample count; then the generation server generates the first random orthogonal matrix according to the total sample count. The number of rows and columns of the first random orthogonal matrix are fixed values, which are the sample counts. For example, the number of rows of the first random orthogonal matrix can be set to the sample count of one data owner, and the number of columns of the first random orthogonal matrix can be set to the number of data owners.

[0035] Figure 2 This is a flowchart of a method for generating a first random orthogonal matrix according to an embodiment of this disclosure. Figure 2 As shown, in some implementations, generating the first random orthogonal matrix may include the following steps:

[0036] Step S210: Create a 0 matrix and randomly generate a random number matrix of size a*a, where a is a preset constant.

[0037] Step S220: Perform QR decomposition on the random number matrix to obtain an orthogonal submatrix, and write the orthogonal submatrix from the (i*a, i*a) position of the 0 matrix, where 0≤i≤a.

[0038] Step S230: Increment the counter variable i and repeat the above steps until no new a*a orthogonal submatrix can be written into the 0 matrix, thus obtaining the first random orthogonal matrix.

[0039] It should be noted that the increment counter variable i is a positive integer, the initial value of the increment counter variable i can be set to 0, and the increment counter variable i will increase sequentially as the orthogonal submatrix is ​​written into the 0 matrix.

[0040] Specifically, the generation server first creates a 0 matrix, whose number of rows and columns is the same as the number of rows and columns required for the first random orthogonal matrix. For example, if the first random orthogonal matrix needs to be configured as a 100*100 orthogonal matrix based on the total number of samples in the original data, then the 0 matrix can also be set to a 100*100 matrix.

[0041] After the server creates a 0 matrix, it then randomly generates a random number matrix of size a*a. 'a' is a preset constant, which can be set according to the total number of samples in the original data. For example, if the total number of samples is 1000, then 'a' can be set to 50, that is, a random number matrix of size 50*50 will be generated.

[0042] After generating the random number matrix, perform QR decomposition on it to obtain an orthogonal submatrix. It's important to note that QR decomposition decomposes the random number matrix into the product of an orthogonal identity matrix and an upper triangular matrix.

[0043] After obtaining the orthogonal submatrix, write the orthogonal submatrix from position (i*a, i*a) of the aforementioned 0 matrix. After the orthogonal submatrix is ​​written into the 0 matrix, the increment counter variable i increases with the number of times the orthogonal submatrix is ​​written; after the increment counter variable i increases, repeat the above steps until no new a*a orthogonal submatrix can be written into the 0 matrix, thus obtaining the first random orthogonal matrix.

[0044] If the diagonal of the 0 matrix can be further filled with orthogonal submatrices, but not enough to fill an a*a orthogonal submatrix, then a smaller random number matrix can be generated. This random number matrix can then be decomposed using QR decomposition to obtain an even smaller orthogonal submatrix. This smaller orthogonal submatrix is ​​then written into the (i*a, i*a) position of the 0 matrix to obtain the first random orthogonal matrix. It should be noted that the newly generated orthogonal submatrix must ensure that after it fills the 0 matrix, the 0 matrix cannot continue to fill the diagonal with other new orthogonal submatrices.

[0045] As an example, let the value of a be 5000 and the initial value of i be 0. The server randomly generates a 5000*5000 random number matrix. This matrix is ​​then subjected to QR decomposition to obtain an orthogonal submatrix. This orthogonal submatrix is ​​written from position (0*5000, 0*5000) of the zero matrix, which is the (0, 0) position of the zero matrix. After writing the orthogonal submatrix from position (0*5000, 0*5000), the counter variable i is incremented to 1. The orthogonal submatrix is ​​then written from position (1*5000, 1*5000) of the zero matrix. This process is repeated, continuously filling the zero matrix with 5000*5000 orthogonal submatrixes along its diagonal, until the zero matrix can no longer hold another 5000*5000 orthogonal submatrix and can no longer be filled with orthogonal submatrixes of other sizes along its diagonal. This yields the first random orthogonal matrix.

[0046] If the zero matrix cannot be filled with another 5000*5000 orthogonal submatrix, but can be filled with orthogonal submatrices of other sizes, the generator server generates a smaller random number matrix to obtain a new orthogonal submatrix smaller than the 5000*5000 orthogonal submatrix, and writes the new orthogonal submatrix from the (i*a, i*a) position of the zero matrix, where i is the incremented value. When the zero matrix cannot be filled with any other orthogonal submatrix of other sizes along its diagonal after this new orthogonal submatrix is ​​filled, the first random orthogonal matrix is ​​obtained.

[0047] The first random seed is generated by the generation server and is used when the data owner needs to generate the second random orthogonal matrix. In this embodiment, the second random orthogonal matrix is ​​represented by the letter E, and the number of rows and columns of the second random orthogonal matrix is ​​equal to the number of samples in the original data of each data owner.

[0048] Specifically, all data owners obtain the same first random seed from the generation server, and then use this first random seed to generate a second random orthogonal matrix. The specific method for generating the second random orthogonal matrix is ​​similar to the method for generating the first random orthogonal matrix described above, and will not be repeated here.

[0049] It should be noted that both the first and second random orthogonal matrices are used by the data owners to encrypt their original data. After the generation server generates the first random orthogonal matrix, it needs to be processed before it can be output to the data owners for further encryption of the original data.

[0050] In some implementations, before sending the encryption key to at least one data owner, the method further includes: dividing the first random orthogonal matrix into at least one first submatrix based on the amount of original data from the at least one data owner.

[0051] Specifically, after generating the first random orthogonal matrix, the generation server divides the first random orthogonal matrix according to the amount of original data (i.e., the number of samples in the original data) from at least one data owner. When dividing the first random orthogonal matrix, the generation server needs to ensure that the number of rows in the first submatrix after division is the same as the amount of original data from the corresponding data owner. For example, if data owner 1 has 100 original data items and data owner 2 has 200 original data items, then the generation server can divide the first random orthogonal matrix into a 100*1 first submatrix and a 200*1 first submatrix.

[0052] Thus, the data owner can obtain a first sub-matrix from the generating server that matches the amount of their original data, facilitating subsequent encryption of the original data.

[0053] Step S120: Send the encryption key to at least one data owner so that at least one data owner can use the encryption key to encrypt their own original data to obtain encrypted data.

[0054] After obtaining the first submatrix and the first random seed, the first submatrix is ​​sent to the corresponding data owner, and the first random seed is sent to each data owner, so that the data owner can use the first submatrix and the first random seed to encrypt its own original data and obtain encrypted data.

[0055] In some embodiments, the original data needs to be standardized before the data owner encrypts it using an encryption key.

[0056] Figure 3 This is a flowchart of a method for standardizing raw data according to an embodiment of this disclosure. Figure 3 As shown, the method includes the following steps:

[0057] Step S310: Receive the random perturbation vectors sent by each data owner, and calculate the mean vector of the original data based on the random perturbation vectors.

[0058] Step S320: Send the mean vector to each data owner so that each data owner can standardize their own raw data based on the mean vector.

[0059] It should be noted that the random perturbation vector of the data owner is obtained based on its own original data and random vector, and the random vector of the data owner is generated based on a second random seed determined in cooperation with other data owners.

[0060] Specifically, each data owner reads the raw data from their local disk; based on the raw data, the data owner calculates the sum of all sample values ​​for each feature in the raw data. The sum of all sample values ​​for all features in a data owner's raw data is represented by a matrix X. i Let i represent the i-th data owner.

[0061] The data owners secretly negotiate a second random seed between each other. This second random seed is then generated by the generation server after negotiation between the two data owners. Each pair of data owners will negotiate a second random seed, and when these two data owners need to use the second random seed, they will obtain the negotiated second random seed from the generation server.

[0062] In this embodiment, the second random seed is denoted as S. i,j Where i and j represent the i-th and j-th data owners, that is, the second random seed S negotiated by the i-th and j-th data owners. i,j 1≤i≤j≤K, if i>j, then use S j,i This represents the second random seed.

[0063] The data owner generates random vectors using a second random seed obtained through negotiation. Random vectors representing all features of the original data are represented by a random vector matrix M. i,j The random vector matrix M represents... i,j The number of rows and columns represents the number of samples in the original data, and its dimension can be n*d.

[0064] In some implementations, the random vector matrix can be generated using a second random seed by using a hash algorithm as a cryptographically secure pseudo-random number generator (CSPRNG). The hash function H can be SHA-256.

[0065] Specifically, the second random seed S i,j and random vector matrix M i,j The dimension is input into the hash function H, and n*d random numbers r are calculated using the hash function H, i.e., r = H(S i,j The binary random number is then converted into data of a specific data structure (such as a floating-point number) according to rules; the data is then rearranged to obtain an n*d dimensional symmetric random vector matrix M.i,j .

[0066] After obtaining the random vector matrix, the data owner uses the random vector matrix M i,j The random perturbation vector is calculated. In this embodiment, the random perturbation vector is represented by X. i ' represents the sum of all sample values ​​under all features of the original data. i With random vector matrix M i,j Add, i.e., X i =X i +Σ i≠j sign(i,j)M i,j Where, if i > j, sign(i,j) = 1; if i < j, sign(i,j) = -1. Thus, the random perturbation vector X is obtained. i ′.

[0067] After each data owner calculates its random perturbation vector, each data owner sends the random perturbation vector to the generation server. The generation server receives the random perturbation vectors from all parties and uses them to calculate the mean vector of the original data. It should be noted that the original data referred to here is the original data provided by all data owners.

[0068] In some implementations, the generation server needs to calculate the sum of all random perturbation vectors before calculating the mean vector of the original data. In this embodiment, the sum of the random perturbation vectors is denoted by X'. The sum of the random perturbation vectors X' can be calculated using the following formula:

[0069] X' = ​​X1' + X2' + ... + X K '

[0070] After calculating the sum X' of the random disturbance vectors, the mean vector of the original data is then calculated. In this embodiment, the mean vector is denoted by P. The mean vector P of the original data can be calculated using the following formula:

[0071] P = X' / n

[0072] Where n is the number of samples in the original data.

[0073] After obtaining the mean vector, the generation server sends it to each data owner. The data owners use the received mean vector to standardize their original data, and then save the standardized data to their local disk. The original data can be standardized using the following formula:

[0074]

[0075] Among them, Wi The original data after standardization for the i-th data owner. The original data belongs to the i-th data owner.

[0076] In some implementations, after standardizing the original data, the data owner encrypts their standardized original data using an encryption key. After encrypting the standardized original data, the data owner obtains an encryption matrix (i.e., encrypted data). The encryption matrix can be obtained using the following formula:

[0077]

[0078] in, Let Q be the encryption matrix of the i-th data owner. i Let E be the first submatrix of the i-th data owner. i Let be the second random orthogonal matrix of the i-th data owner.

[0079] Step S130: Receive encrypted data sent by at least two data owners, and aggregate the encrypted data from at least two data owners to obtain a relation matrix.

[0080] As an example, K is set to 3, and the three data owners are denoted as A, B, and C. The server can send encryption keys to A, B, and C respectively, and receive encrypted data from A, B, and C, where A, B, and C all use the current encryption key for encryption; the server can also send encryption keys to A respectively, and receive encrypted data from A and B, where B knows A's encryption method and can also generate encrypted data; the server can also send encryption keys to A, B, and C respectively, and receive encrypted data from A, B, and C, where only A encrypts using the current encryption key, while B and C retain previously encrypted data and do not encrypt it again.

[0081] After receiving the encrypted data (i.e., the encryption matrix), the data owner sends the encrypted data to the decomposition server. The decomposition server receives encrypted data from at least two data owners and performs aggregation processing on the encrypted data. In some implementations, the encrypted data can be aggregated using the following formula:

[0082]

[0083] in, This is a relation matrix, where n is the number of samples in the original data. Let W be the encryption matrix of the i-th data owner, and W be the normalized original data.

[0084] Step S140: Obtain the first intermediate matrix and the first perturbation matrix based on the relation matrix, and send the first intermediate matrix and the first perturbation matrix to K data owners so that the K data owners can obtain the second perturbation matrix based on the first perturbation matrix and their own original data.

[0085] Before obtaining the first intermediate matrix and the first perturbation matrix from the relation matrix, the decomposition server needs to estimate the variance of the relation matrix and the residuals to obtain the matrix variance and the residual variance, and then obtain the first intermediate matrix and the first perturbation matrix through the matrix variance and the residual variance.

[0086] In some implementations, the variance is estimated based on the relation matrix using a first preset constrained maximum likelihood estimation algorithm and a second preset constrained maximum likelihood estimation algorithm to obtain the matrix variance and residual variance. Specifically, the first preset constrained maximum likelihood estimation algorithm is used in the first round of iterations, while the second preset constrained maximum likelihood estimation algorithm is used in other rounds of iterations.

[0087] In this embodiment, the matrix variance is represented by σ. g The residual variance is represented by σ. ∈ This indicates that before optimizing the matrix variance and residuals, they need to be initialized. The variance of y (i.e., the phenotype) can be used as the initial value. For example,

[0088] Specifically, the matrix variance and residual variance are continuously optimized through multiple rounds of iteration to obtain the final required matrix variance and residual variance. The optimization direction for the matrix variance and residual variance can be found first using a first-preset constrained maximum likelihood estimation algorithm, and then a second-preset constrained maximum likelihood estimation algorithm can be used for subsequent iterations.

[0089] The first pre-defined constrained maximum likelihood estimation algorithm can use the EM-REML (Expectation Maximization Restricted Maximum Likelihood) algorithm. The EM-REML algorithm is used for the first round of iterations on the matrix variance and residual variance to find the optimization direction for these two variances. The following formula can be used for the first round of iterations on the matrix variance and residual variance:

[0090]

[0091]

[0092] in, The variance of the matrix after the first round of iterations. Let P be the residual variance after the first iteration, and P be an intermediate variable. X g Let A be the original data matrix, I be the relation matrix, y be the identity matrix, y be the phenotypic matrix, and tr() be the trace of the matrix.

[0093] The second pre-defined constrained maximum likelihood estimation algorithm can use the AI-REML (Average Information Restricted Maximum Likelihood) algorithm. The AI-REML algorithm is used to perform other rounds of iteration on the matrix variance and residual variance until the final matrix variance and residual variance are obtained.

[0094] It should be noted that the loss function of the AI-REML algorithm is: Let θ = σ g ,σ ∈ Then, other rounds of iterations can be calculated using the following formula:

[0095]

[0096] Where t is the number of iterations using the AI-REML algorithm, and AI is the average information matrix, defined as follows:

[0097]

[0098] Where y is the phenotypic trait matrix, r is the number of features in the original data, and if the original data is gene data, r can be the number of random genetic effects.

[0099] also, It can be calculated using the following formula:

[0100]

[0101] Therefore, after multiple iterations, the final value of θ can be obtained, and thus the matrix variance σ can be obtained. g and residual variance σ ∈ .

[0102] After obtaining the matrix variance and residual variance, the first intermediate matrix and the first perturbation matrix are obtained using these variances. In some implementations, the first intermediate matrix and the first perturbation matrix can be obtained using the following formula:

[0103]

[0104]

[0105]

[0106]

[0107] V = sum(A*σ) g ,I*σ ∈ ),

[0108] Where, β n This is the first intermediate matrix. The transpose of the original data matrix, where P is the mean vector. Let R1 be the first perturbation matrix, and σ be the pseudo-random number. g Let σ be the matrix variance. ∈ Let A be the residual variance, Y be the relation matrix, and I be the phenotypic matrix.

[0109] In order to be in The Kolesky decomposition on V should be noted, because V -1 Since it is a symmetric positive definite matrix, in The Kolesky decomposition always exists.

[0110] After obtaining the first intermediate matrix and the first perturbation matrix, the decomposition server sends them to each data owner. Upon receiving the first intermediate matrix and the first perturbation matrix, each data owner multiplies the first perturbation matrix by the transpose of its own original data matrix to obtain the second perturbation matrix. In this embodiment, the second perturbation matrix is ​​used... Let i represent the i-th data owner.

[0111] Step S150: Receive the second perturbation matrix sent by K data owners, obtain the target diagonal matrix based on the second perturbation matrix, and send the target diagonal matrix to each data owner so that the K data owners can obtain the distribution test value of their own original data based on the target diagonal matrix and the first intermediate matrix.

[0112] The data owner obtains the second perturbation matrix. Then, the second perturbation matrix is ​​sent to the decomposition server; the decomposition server obtains the target diagonal matrix by receiving the second perturbation matrices sent by each data owner.

[0113] In some implementations, the target diagonal matrix is ​​obtained by the following formula:

[0114]

[0115] Where, β d For the target diagonal matrix, Let m be the original data matrix of the i-th data owner, and m be the number of SNPs. R1 is a diagonal random matrix, and R2 is a third random orthogonal matrix. It should be noted that R3 can be any random orthogonal matrix.

[0116] Obtain the target diagonal matrix β d Then, the decomposition server distributes the target diagonal matrix to each data owner; each data owner calculates the distribution test value of its own original data based on the received target diagonal matrix and the first intermediate matrix.

[0117] In some implementations, the distribution test values ​​may include a Beta value and a chi-square value. The Beta value is represented by β, and the chi-square value by χ². 2 express.

[0118] The Beta value and chi-square value can be obtained using the following formula:

[0119]

[0120]

[0121] Where E is the second random orthogonal matrix, β n Let β be the first intermediate matrix mentioned above. d Let R1 be the target diagonal matrix, and R2 be a diagonal random matrix.

[0122] After each data owner calculates the distribution test value of the MLM model, they store the distribution test value on their local disk.

[0123] Thus, the first random orthogonal matrix and the first random seed are generated by the generation server; then, the data owners negotiate the second random seed in pairs, and obtain the second random orthogonal matrix based on the second random seed; the data owners encrypt their own original data using the obtained first and second random orthogonal matrices, and then send the encrypted data to the decomposition server, thus ensuring the privacy of the original data transmission.

[0124] The encrypted data is aggregated by the decomposition server to obtain a relation matrix; then, a first perturbation matrix and a first intermediate matrix are obtained based on the relation matrix; the decomposition server then sends the first perturbation matrix and the first intermediate matrix to each data owner, so that the data owner adds perturbation to its own original data according to the first perturbation matrix to obtain a second perturbation matrix; a target diagonal matrix is ​​obtained based on the second perturbation matrix; and the data owner finally obtains the distribution test value of its own original data based on the target diagonal matrix and the first intermediate matrix.

[0125] This approach enables cross-institutional data sharing and fusion while ensuring data privacy and security, thereby expanding the sample size of the original data and improving the accuracy of distribution test values ​​in the MLM model. Furthermore, since the random matrices used in this embodiment are all orthogonal matrices, using orthogonal matrices reduces computation and communication overhead, not only maintaining data transmission security but also improving the computational efficiency of distribution test values.

[0126] Figure 4 This is a flowchart of a data multi-party collaborative processing method according to a second embodiment of this disclosure. The executing entity in this embodiment can be the data owner, such as... Figure 4 As shown, the method includes the following steps:

[0127] Step S410: Receive the first intermediate matrix and the first perturbation matrix sent by the server, and obtain the second perturbation matrix based on the first perturbation matrix and its own original data.

[0128] Step S420: Send the second perturbation matrix to the server so that the server can obtain the target diagonal matrix based on the second perturbation matrix.

[0129] Step S430: Receive the target diagonal matrix sent by the server, and obtain the distribution test value of its own original data based on the target diagonal matrix and the first intermediate matrix.

[0130] Therefore, by receiving the first intermediate matrix and the first perturbation matrix sent by the server, and obtaining the second perturbation matrix based on the first perturbation matrix and its own original data, the system can achieve cross-institutional data sharing and fusion, and complete MLM modeling while ensuring data privacy and security.

[0131] In some implementations, the method further includes: receiving an encryption key sent by a server, and using the encryption key to encrypt its own original data to obtain encrypted data; sending the encrypted data to the server so that the server can aggregate the encrypted data of K data owners to obtain a relation matrix, and obtain a first intermediate matrix and a first perturbation matrix based on the relation matrix.

[0132] In some implementations, the encryption key includes a first random orthogonal matrix and a first random seed. The server further divides the first random orthogonal matrix into K first sub-matrices according to the amount of original data of each data owner. Receiving the encryption key sent by the server includes receiving the corresponding first sub-matrix sent by the server and receiving the first random seed.

[0133] In some implementations, the method further includes: generating a random vector based on a second random seed determined in cooperation with other data owners, and generating a random perturbation vector based on its own original data and the random vector; sending the random perturbation vector to a server so that the server can calculate the mean vector of the original data based on the random perturbation vectors sent by each data owner; receiving the mean vector sent by the server, and standardizing its own original data based on the mean vector; wherein encrypting its own original data using an encryption key includes: encrypting its own standardized original data using an encryption key.

[0134] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific process of the second data multi-party joint processing method described herein can be referred to the corresponding process in the aforementioned embodiment of the first data multi-party joint processing method, and will not be repeated here.

[0135] To implement the above embodiments, this disclosure also proposes a server, which includes a memory and a processor. The memory stores a computer program, and the processor executes the program to implement the first data multi-party joint processing method as described above.

[0136] Figure 5 This is a structural block diagram of a server according to an embodiment of the present disclosure.

[0137] like Figure 5 As shown, Figure 5 The server 500 shown includes a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, for example, via a bus 502. Optionally, the server 500 may also include a transceiver 504. It should be noted that in practical applications, the transceiver 504 is not limited to one type, and the structure of this server 500 does not constitute a limitation on the embodiments of this disclosure.

[0138] Processor 501 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 501 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0139] Bus 502 may include a pathway for transmitting information between the aforementioned components. Bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 502 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0140] The memory 503 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0141] The memory 503 is used to store application code that executes the present disclosure scheme, and its execution is controlled by the processor 501. The processor 501 is used to execute the application code stored in the memory 503 to implement the content shown in the foregoing method embodiments.

[0142] Figure 5 The server 500 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0143] To implement the above embodiments, this disclosure also proposes a terminal device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the program to implement the second data multi-party joint processing method as described above. It should be noted that the structure of the terminal device can refer to the structure of the server described above.

[0144] To implement the above embodiments, this disclosure also proposes a multi-party data processing system, which includes the server and K terminal devices proposed in the above embodiments, wherein K is an integer greater than or equal to 2.

[0145] In addition, it should be noted that other components and functions of the data multi-party joint processing system in the embodiments of this disclosure are known to those skilled in the art, and will not be described in detail here in order to reduce redundancy.

[0146] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0147] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0148] In the description of this disclosure, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this disclosure and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this disclosure.

[0149] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0150] In this disclosure, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this disclosure according to the specific circumstances.

[0151] In this disclosure, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0152] Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for multi-party collaborative data processing, characterized in that, The original data is distributed among K data owners, where K is an integer greater than or equal to 2. The method includes: Generate encryption key; The encryption key is sent to at least one of the data owners, so that at least one of the data owners can use the encryption key to encrypt their own original data to obtain encrypted data; Receive encrypted data sent by at least two data owners, and aggregate the encrypted data from at least two data owners to obtain a relation matrix; The first intermediate matrix and the first perturbation matrix are obtained based on the relationship matrix, and the first intermediate matrix and the first perturbation matrix are sent to the K data owners so that the K data owners can obtain the second perturbation matrix based on the first perturbation matrix and their own original data. The system receives a second perturbation matrix sent by the K data owners, obtains a target diagonal matrix based on the second perturbation matrix, and sends the target diagonal matrix to each of the data owners so that the K data owners can obtain the distribution test value of their own original data based on the target diagonal matrix and the first intermediate matrix. The encrypted data is aggregated using the following formula: in, Let n be the relation matrix, and n be the number of samples in the original data. , For the encrypted data of the i-th data owner, Q i Let E be the first submatrix corresponding to the i data owners. i The second random orthogonal matrix is ​​generated by the i-th data owner based on the first random seed in the encryption key, and W is the normalized original data; The first intermediate matrix and the first perturbation matrix are obtained by the following formula: ， = ， ， ， ， in, This is the first intermediate matrix. Original data matrix The transpose of the matrix, It is the mean vector. This is the first perturbation matrix. It is a pseudo-random number. For the matrix variance, Let Y be the residual variance, Y be the phenotypic matrix, and I be the identity matrix.

2. The data multi-party joint processing method according to claim 1, characterized in that, The encryption key includes a first random orthogonal matrix and a first random seed. Before sending the encryption key to at least one of the data owners, the method further includes: Based on the original data quantity of at least one of the data owners, the first random orthogonal matrix is ​​divided into at least one first sub-matrix; The step of sending the encryption key to at least one of the data owners includes: The first submatrix is ​​sent to the corresponding data owner, and the first random seed is sent to at least one of the data owners respectively.

3. The data multi-party joint processing method according to claim 2, characterized in that, Generating the first random orthogonal matrix includes: Create a 0 matrix and randomly generate a matrix of size a. A random number matrix of type a, where a is a preset constant; Perform QR decomposition on the random number matrix to obtain an orthogonal submatrix, and then separate the orthogonal submatrix from the (i) of the 0 matrix. a,i a) Write the position, where 0 ≤ i ≤ a; Increment the counter variable i and repeat the above steps until no new a can be written into the zero matrix. A random orthogonal matrix is ​​obtained by using an orthogonal matrix a.

4. The data multi-party joint processing method according to claim 2, characterized in that, The method further includes: The system receives random perturbation vectors sent by each of the data owners and calculates the mean vector of the original data based on the random perturbation vectors. The random perturbation vector of each data owner is obtained based on its own original data and random vector, and the random vector of the data owner is generated based on a second random seed determined in cooperation with other data owners. The mean vector is sent to each of the data owners so that each of the data owners can standardize their own raw data based on the mean vector. The data owner uses the encryption key to encrypt its own standardized raw data.

5. The data multi-party joint processing method according to claim 1, characterized in that, The variance is estimated based on the relation matrix using the first and second preset constrained maximum likelihood estimation algorithms to obtain the matrix variance and residual variance. The first preset constrained maximum likelihood estimation algorithm is used in the first iteration, while the second preset constrained maximum likelihood estimation algorithm is used in other iterations.

6. The data multi-party joint processing method according to claim 1, characterized in that, The target diagonal matrix is ​​obtained using the following formula: = ， in, Let be the target diagonal matrix. Let m be the original data matrix of the i-th data owner, and m be the number of SNPs. , , is a diagonal random matrix. It is the third random orthogonal matrix.

7. The data multi-party joint processing method according to claim 1, characterized in that, The distribution test values ​​include the Beta value and the chi-square value.

8. A method for multi-party collaborative data processing, characterized in that, The original data is distributed among K data owners, where K is an integer greater than or equal to 2. The method includes the following steps: The server receives a first intermediate matrix and a first perturbation matrix, and obtains a second perturbation matrix based on the first perturbation matrix and its own original data. The first intermediate matrix and the first perturbation matrix are obtained by the server using the data multi-party joint processing method as described in any one of claims 1-7. The second perturbation matrix is ​​sent to the server so that the server obtains the target diagonal matrix based on the second perturbation matrix; The system receives the target diagonal matrix sent by the server and obtains the distribution test value of its own original data based on the target diagonal matrix and the first intermediate matrix.

9. The data multi-party joint processing method according to claim 8, characterized in that, The method further includes: Receive the encryption key sent by the server, and use the encryption key to encrypt its own original data to obtain encrypted data; The encrypted data is sent to the server so that the server can aggregate the encrypted data of the K data owners to obtain a relation matrix, and obtain the first intermediate matrix and the first perturbation matrix based on the relation matrix.

10. The data multi-party joint processing method according to claim 9, characterized in that, The encryption key includes a first random orthogonal matrix and a first random seed. The server further divides the first random orthogonal matrix into K first sub-matrices based on the amount of original data from each data owner. The encryption key sent by the receiving server includes: Receive the corresponding first sub-matrix sent by the server, and receive the first random seed.

11. The data multi-party joint processing method according to claim 10, characterized in that, The method further includes: A random vector is generated based on a second random seed determined in cooperation with other data owners, and a random perturbation vector is generated based on its own original data and the random vector. The random perturbation vector is sent to the server so that the server can calculate the mean vector of the original data based on the random perturbation vectors sent by each of the data owners. Receive the mean vector sent by the server, and standardize its own raw data according to the mean vector; The step of encrypting its own original data using the encryption key includes: encrypting its own standardized original data using the encryption key.

12. A server, characterized in that, include: A memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, it implements the data multi-party joint processing method as described in any one of claims 1-7.

13. A terminal device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, it implements the data multi-party joint processing method as described in any one of claims 8-11.

14. A multi-party collaborative data processing system, characterized in that, The system includes: a server as described in claim 12 and K terminal devices as described in claim 13, wherein K is an integer greater than or equal to 2.

Images