Multi-client data processing method, system, device, medium and product
By standardizing client data and transforming principal component analysis and transforming, and clustering with Wastherstein distance and Martha distance, data heterogeneity problem in traditional federated learning is solved, processing efficiency is improved and personalized prediction is supported.
Patent Information
- Application Number
- CN202410843554.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-27
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-06-27
AI Technical Summary
In traditional federated learning, due to data heterogeneity, non-independent and homogeneous distribution of data are ignored, resulting in inefficiency and inability to produce personalized predictions.
By standardizing and principal component analysis transformation of each client data, clustering with Wastherstein distance and Martha distance, federated learning is performed separately to improve efficiency.
The similarity judgment and quantification of the distribution of multi-client data is realized, the processing efficiency of non-independent and homogeneous data is improved, and personalized prediction is supported.
Smart Images

Figure CN118396137B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of communication technology, and in particular to a multi-client data processing method, system, device, medium and product. Background Art
[0002] With the rapid popularization of smart terminal devices, a large amount of personal information data is collected by various network platforms. Traditional machine learning requires the client's information data to be collected to the central server, which can easily lead to the leakage of the client's privacy data. Federated learning, as a distributed machine learning model, does not require the data to leave the local client, thereby protecting the privacy of the data. In federated learning, since the data comes from a variety of terminal devices, data heterogeneity problems will naturally arise, that is, the data is non-independent and identically distributed (non-IID). In applications such as recommendation systems and personalized advertising, it is particularly important to utilize data heterogeneity.
[0003] In the related technical solution, the data of all clients are directly brought into the same global model for training. After the training is completed, all clients also use the same model for prediction. This approach directly processes non-IID data as independent and identically distributed data, ignoring the differences between multi-client data, and thus cannot produce some personalized predictions. In addition, the stochastic gradient descent method is designed for independent and identically distributed data, and there will be a problem of low efficiency when processing non-IID data. Summary of the invention
[0004] The purpose of the present invention is to provide a multi-client data processing method, system, device, medium and product, which can use the Wasserstein distance combined with the Mahalanobis distance to directly quantify the similarity between multi-client data distributions, and has high data processing efficiency.
[0005] In order to solve the above technical problems, the present invention provides a multi-client data processing method, the method comprising:
[0006] Each client standardizes its own data and uses the principal component analysis method to transform the standardized data;
[0007] Each client sorts each component in the transformed data, and extracts a set number of values from each sorted component at a set interval;
[0008] Each client sends the extracted values to the server, so that the server clusters all clients using the extracted values combined with Wasserstein distance and Mahalanobis distance;
[0009] After all clients are clustered, clients in different categories perform federated learning separately.
[0010] In a first aspect, in the multi-client data processing method provided by the present invention, each client standardizes its own data, including:
[0011] Each client obtains the sum of all its own data;
[0012] Each client sends the sum of all its own data and the number of its own data to the server, so that the server calculates the average value of all the clients' data and sends the average value of the data to each client;
[0013] Each client calculates a local partial variance based on the data average value and sends it to the server, so that the server calculates the data standard deviation and sends it to each client;
[0014] Each client standardizes its own data according to the data standard deviation.
[0015] On the other hand, in the multi-client data processing method provided by the present invention, each client uses the following formula to obtain the sum of all its own data:
[0016] ;
[0017] in, For the The sum of all data of the clients, The vector is represented in the form of The form of Client data No. The rows are converted into column vectors. Indicates Client data No. The value of row 1, column 1, Indicates Client data No. Line The value of the column, , For the The number of data of each client. is the dimension of all data, represents the summation symbol, Indicates transpose.
[0018] On the other hand, in the multi-client data processing method provided by the present invention, the following formula is used to calculate the average value of the data of all clients:
[0019] ;
[0020] ;
[0021] ;
[0022] in, Represents the total data of all clients. , is the total number of clients, is the number of data of the first client, For the The number of data of each client. is the total number of data of all clients, It is the average value of data of all clients.
[0023] On the other hand, in the multi-client data processing method provided by the present invention, each client calculates the local partial variance using the following formula:
[0024] ;
[0025] ;
[0026] in, For the The local partial variance of the client, For the The local partial variance of the client Quantity, , For the Client data In line 1 The value of the column, For the Client data In the Line The value of the column, The average value of the data No. A quantity.
[0027] On the other hand, in the multi-client data processing method provided by the present invention, the data standard deviation is calculated using the following formula:
[0028] ;
[0029] ;
[0030] in, is the standard deviation of the data, The data variance is obtained by adding the local partial variances of all clients. is the data variance No. A quantity.
[0031] On the other hand, in the multi-client data processing method provided by the present invention, each client standardizes its own data using the following formula according to the data standard deviation:
[0032] ;
[0033] ;
[0034] ;
[0035] in, For the Client data In the Line The data obtained after the column values are standardized. For the Client data In the Line The value of the column, is the standard deviation of the data No. Quantity, For the After client standardization Items of data, For the The data is standardized by the client.
[0036] On the other hand, in the multi-client data processing method provided by the present invention, the principal component analysis method is used to transform the standardized data, including:
[0037] The total number of data sent by the receiving server;
[0038] According to the total number of data and the standardized data, a local covariance matrix is calculated and sent to the server, so that the server calculates the overall covariance matrix, performs singular value decomposition on the overall covariance matrix, selects several eigenvectors corresponding to the largest eigenvalues and sends them to each client;
[0039] Each client performs transformation processing on the standardized data according to the selected feature vector.
[0040] On the other hand, in the multi-client data processing method provided by the present invention, the local covariance matrix is calculated using the following formula:
[0041] ;
[0042] in, is the local covariance matrix, is the total number of data of all clients, For the After client standardization Items of data, , For the The number of data of each client. For the The standardized data of each client, Indicates transpose.
[0043] On the other hand, in the multi-client data processing method provided by the present invention, the server calculates the overall covariance matrix, performs singular value decomposition on the overall covariance matrix, selects several eigenvectors corresponding to the largest eigenvalues and sends them to each client, including:
[0044] The server calculates the overall covariance matrix ;
[0045] For the population covariance matrix Perform singular value decomposition and get ,in is a diagonal matrix that satisfies , is an orthogonal matrix;
[0046] Select Before OK Sent to each client.
[0047] On the other hand, in the multi-client data processing method provided by the present invention, each client uses the following formula to transform the standardized data according to the selected feature vector:
[0048] ;
[0049] ;
[0050] in, For the The client is normalized to the The data obtained after the data is transformed. For the The data obtained after a client transforms the standardized data.
[0051] On the other hand, in the multi-client data processing method provided by the present invention, each client sorts each component in the transformed data, and extracts a set number of values from each sorted component at a set interval, including:
[0052] No. The client will transform the processed data Quantity Sort by
[0053] Extract the values, among which , , The set quantity is as described;
[0054] If the extracted value is , No. All values extracted by the client are:
[0055] ;
[0056] in, For the All values extracted by the client, is the dimension of all extracted values, , , is the dimension of all data.
[0057] On the other hand, in the multi-client data processing method provided by the present invention, all clients are clustered using the extracted values in combination with Wasserstein distance and Mahalanobis distance, including:
[0058] Select data from multiple clients as cluster centers;
[0059] Calculate the target distance between each client data and each cluster center obtained by combining Wasserstein distance with Mahalanobis distance;
[0060] Assign each client to the cluster whose cluster center is closest to the target;
[0061] Update cluster center points;
[0062] Repeat the steps of calculating the target distance and updating the cluster center points until the cluster center points no longer change.
[0063] On the other hand, in the multi-client data processing method provided by the present invention, selecting data of multiple clients as cluster centers includes:
[0064] Select The data of each client is used as the cluster center;
[0065] Will The data of cluster centers are recorded as :
[0066] ;
[0067] in, , For the set quantity, , is the dimension of all extracted values, , , is the dimension of all data, is the cluster center data Elements of Indicates transpose.
[0068] On the other hand, in the multi-client data processing method provided by the present invention, the target distance between each client data and each cluster center obtained by combining the Wasserstein distance with the Mahalanobis distance is calculated using the following formula:
[0069] ;
[0070] in, For the Client data and The target distance between cluster centers, It is client data Elements of is a diagonal matrix No. diagonal elements.
[0071] On the other hand, in the multi-client data processing method provided by the present invention, updating the cluster center point includes:
[0072] Get the average value of client data belonging to each category;
[0073] The average value of the client data belonging to each category is used as the new cluster center point.
[0074] On the other hand, in the multi-client data processing method provided by the present invention, the following formula is used to obtain the average value of the client data belonging to each category:
[0075] ;
[0076] in, For the The client data of the class, For the A collection of client indexes of the class, Representing a collection The number of elements, For the The average value of client data for the class.
[0077] On the other hand, in the multi-client data processing method provided by the present invention, clients in different categories perform federated learning respectively, including:
[0078] Each client receives a global model of the corresponding category established by the server; all clients in each category receive the same global model;
[0079] Each client trains its own global model and uploads the weights and gradient parameters to the server, so that the server aggregates the model parameters within the corresponding category to obtain a new global model of the corresponding category;
[0080] Until the loss function of the corresponding category converges or the number of iterations reaches the set number, the trained global model of the corresponding category is output;
[0081] Use the trained global model of the corresponding category to classify or detect the task to be processed.
[0082] In order to solve the above technical problems, the present invention also provides a multi-client data processing system, the system comprising a plurality of clients and at least one server;
[0083] Each client is used to standardize its own data and transform the standardized data using the principal component analysis method; it is also used to sort each component in the transformed data, and extract a set number of values from each sorted component at a set interval, and send the extracted values to the server;
[0084] The server is used to cluster all clients using the extracted values combined with Wasserstein distance and Mahalanobis distance;
[0085] Clients in different categories are used to perform federated learning separately.
[0086] In order to solve the above technical problems, the present invention further provides a multi-client data processing device, the device comprising:
[0087] Memory for storing computer programs;
[0088] A processor is used to implement the steps of the above-mentioned multi-client data processing method when executing the computer program.
[0089] In order to solve the above technical problem, the present invention also provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above multi-client data processing method are implemented.
[0090] In order to solve the above technical problem, the present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the above multi-client data processing method when executed by a processor.
[0091] It can be seen from the above technical scheme that the present invention provides a multi-client data processing method, which includes: each client standardizes its own data, and uses the principal component analysis method to transform the standardized data; each client sorts each component in the transformed data, and extracts a set number of values for each sorted component according to a set interval; each client sends the extracted values to the server, so that the server uses the extracted values, combined with the Wasserstein distance and the Mahalanobis distance to cluster all clients; after all clients are clustered, clients in different categories perform federated learning respectively.
[0092] The beneficial effect of the present invention is that the multi-client data processing method provided by the present invention first standardizes the data of the client itself to reduce calculation errors, and then uses the principal component analysis method to transform the standardized data, so that only the principal component is used to facilitate subsequent calculations, reduce the dimensionality of the data, and avoid leakage of the original data. Then, the sorted extracted values are combined with the Wasserstein distance and the Mahalanobis distance to cluster all clients, which provides a judgment standard for the similarity of data distribution. The Wasserstein distance is combined with the Mahalanobis distance to judge the relationship between the distribution of multi-client data, and the similarity between the distribution of multi-client data is directly quantified. After all clients are clustered, the data of the clients in each category can be considered to be independent and identically distributed, so it is more efficient to directly perform federated learning in each category, and the federated learning can perform some personalized predictions. The entire multi-client data processing process improves the processing efficiency of non-independent and identically distributed data.
[0093] In addition, the present invention also provides a corresponding multi-client data processing system, multi-client data processing device, non-volatile storage medium and product for the multi-client data processing method, which have the same or corresponding technical features as the above-mentioned multi-client data processing method and have the same effect as above. BRIEF DESCRIPTION OF THE DRAWINGS
[0094] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0095] Figure 1 A flowchart of a multi-client data processing method provided by an embodiment of the present invention;
[0096] Figure 2 A schematic diagram of the structure of a multi-client data processing device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0097] Traditional machine learning requires all information to be collected to a central server first, and the client's private data is easily leaked. Federated Learning, as a distributed machine learning model, does not require data to leave the local client, thereby protecting the privacy of the data. In federated learning, since the data comes from a variety of terminal devices, data heterogeneity will naturally arise, that is, the data is not independent and identically distributed. In applications such as recommendation systems and personalized advertising, it is particularly important to utilize data heterogeneity, which is beneficial to both users and enterprises. In traditional federated learning, all user data is brought into the same global model for training. After the training is completed, all users also use the same model for prediction. This approach treats non-independent and identically distributed data as independent and identically distributed data, ignoring the differences between multi-client data, so some personalized predictions cannot be generated. In addition, the stochastic gradient descent method is designed for independent and identically distributed data, and there will be a problem of low efficiency when processing non-independent and identically distributed data.
[0098] In order to solve the problems of low efficiency and ignoring differences between multi-client data when processing non-independent and identically distributed data, the present invention provides a multi-client data processing method, system, device, medium and product.
[0099] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.
[0100] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1A flowchart of a multi-client data processing method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes:
[0101] S101. Each client standardizes its own data and uses a principal component analysis method to transform the standardized data.
[0102] It should be noted that each client standardizes its own data, which can eliminate the dimensional influence and numerical size differences between data features, making different features comparable, simplifying the calculation process, reducing calculation errors, and improving the performance of the algorithm.
[0103] Principal Component Analysis (PCA) is a data dimensionality reduction technology, which is mainly used to process high-dimensional data and convert it into low-dimensional data while retaining most of the important information in the data. Specifically, vectors with certain correlations can be recombined into a new set of mutually unrelated vectors with the help of orthogonal transformation, and then a few principal components are derived to achieve the purpose of data dimensionality reduction. The present invention uses the principal component analysis method to transform the standardized data, which has a dimensionality reduction effect on the data, facilitates the subsequent calculation to use only principal components, and avoids the leakage of original data.
[0104] S102: Each client sorts each component in the transformed data, and extracts a set number of values from each sorted component at a set interval.
[0105] In implementation, the present invention sorts each component of the transformed data for each client, and then extracts a set number of values (such as extracting values), and use these extracted values to approximate the distribution that the client data follows.
[0106] S103. Each client sends the extracted value to the server, so that the server clusters all clients by using the extracted value in combination with Wasserstein distance and Mahalanobis distance.
[0107] It should be noted that Wasserstein distance is a method for measuring the difference between two probability distributions, which can be understood as the minimum cost of converting from one probability distribution to another. Compared with other distances that measure the difference between distributions, Wasserstein distance is more stable. Even if the support sets of the two distributions do not overlap or overlap very little, Wasserstein distance can still measure the difference between the distributions well and reflect the distance between the two distributions.
[0108] Mahalanobis distance is a method for measuring the distance between two sample points, which takes into account the sample distribution (i.e. the correlation between various features). Specifically, Mahalanobis distance represents the covariance distance of data and can be considered as an effective method for calculating the similarity between multi-client data. It is not affected by the dimension and can eliminate the interference of correlation between variables.
[0109] In order to directly consider the similarity between data distributions of different clients in federated learning, the present invention combines Wasserstein distance and Mahalanobis distance to form a new distance, which can be called Wasserstein-Mahalanobis distance. The present invention can directly measure the similarity between data distributions of multiple clients through Wasserstein-Mahalanobis distance, that is, establish a standard for directly measuring the similarity of data distributions of multiple clients, and then use this standard to cluster all clients, identify multiple clients with similar distributions, and classify them into one category. After classification, since the distribution distances between multiple clients in the same category are relatively close, their data can be regarded as independent and identically distributed.
[0110] S104: After all clients are clustered, clients in different categories perform federated learning separately.
[0111] It should be noted that federated learning is a distributed machine learning technology, which aims to achieve joint modeling on the basis of ensuring data privacy, security and legal compliance. In the present invention, the data of multiple clients in the same category can be regarded as independent and identically distributed, so federated learning can be directly performed in each category.
[0112] Assume that client A in the first category and client B in the second category have different data, for example, client A in the first category contains user feature data; client B in the second category contains product feature data and annotation data. Since the data of client A in the first category and client B in the second category cannot be directly merged, the present invention can perform federated learning in the first category and federated learning in the second category respectively, so that client A in the first category and client B in the second category can establish a target task model through parameter exchange under an encryption mechanism, that is, without violating data privacy regulations, and the corresponding task classification results or task prediction results can be obtained using the target task model without leaking privacy data.
[0113] It should be noted that the multi-client data processing method provided by the present invention can be understood as a multi-client data processing method based on federated learning specifically related to the field of distributed and privacy protection technology. The method can be used in an environment where sensor devices, independent data center organizations, and edge servers work together.
[0114] In the multi-client data processing method provided by the embodiment of the present invention, the client's own data is first standardized to reduce calculation errors, and then the principal component analysis method is used to transform the standardized data. In this way, only the principal component is used to facilitate subsequent calculations, reduce the dimensionality of the data, and avoid leakage of the original data. Then, the sorted extracted values are combined with the Wasserstein distance and the Mahalanobis distance to cluster all clients, which provides a judgment standard for the similarity of data distribution. The Wasserstein distance is combined with the Mahalanobis distance to judge the relationship between the distribution of multi-client data, and the similarity between the distribution of multi-client data is directly quantified. After all clients are clustered, the data of the clients in each category can be considered to be independent and identically distributed. Therefore, it is more efficient to directly perform federated learning in each category, and the federated learning can perform some personalized predictions. The entire multi-client data processing process improves the processing efficiency of non-independent and identically distributed data.
[0115] Furthermore, in a specific implementation, in the above-mentioned multi-client data processing method provided in an embodiment of the present invention, in step S101, each client standardizes its own data, including: each client obtains the sum of all its own data; each client sends the sum of all its own data and the number of its own data to the server, so that the server calculates the data average of all clients and sends the data average to each client; each client calculates the local partial variance according to the data average and sends it to the server, so that the server calculates the data standard deviation and sends it to each client; each client standardizes its own data according to the data standard deviation.
[0116] In implementation, each client may first obtain the sum of all its own data, and then send the sum of all its own data and the number of its own data to the server; after receiving the sum of all its own data and the number of its own data obtained by each client, the server calculates the data average of all clients and sends the data average to each client; at this time, each client may calculate the local partial variance based on the data average and send it to the server; next, the server may calculate the data standard deviation based on the local partial variance and send it to each client; each client may standardize its own data based on the data standard deviation.
[0117] In order to make the people skilled in the art better understand the various formulas in the technical solution of the present invention, some symbols are first explained: the present invention uses lowercase letters (such as ) represents a scalar, use Express To round up, use Vectors are represented in the form of , using uppercase letters (e.g. ) represents the matrix, using Representation Matrix The inverse matrix of In the present invention, there may be at least one server, N clients, and the number of data of each client is respectively .
[0118] Furthermore, in a specific implementation, in the multi-client data processing method provided in the embodiment of the present invention, when executing step S101, each client may use the following formula to obtain the sum of all its own data:
[0119] ; (1)
[0120] in, For the The sum of all data of the clients, The vector is represented in the form of The form of Client data No. The rows are converted into column vectors. Indicates Client's Items of data, Indicates Client data No. The value of row 1, column 1, Indicates Client data No. Line The value of the column, , For the The number of data of each client. is the dimension of all data, represents the summation symbol, Indicates transpose.
[0121] Among them, The data of the client is .
[0122] In the implementation, each client obtains the sum of the data in its own dataset. The sum of all data of the client corresponds to the above formula (1), that is, summing each component of the data. The sum of all data of the clients and The number of data for each client is Send to the server.
[0123] In specific implementation, in the above multi-client data processing method provided by the embodiment of the present invention, when executing step S101, the server can use the following formula to calculate the average value of the data of all clients:
[0124] ; (2)
[0125] ; (3)
[0126] ; (4)
[0127] in, Represents the total data of all clients. , is the total number of clients, is the number of data of the first client, For the The number of data of each client. is the total number of data of all clients, It is the average value of data of all clients.
[0128] In implementation, upon receiving the The sum of all data of the clients and The number of data for each client is After that, the total data of all clients can be calculated , the total number of data from all clients , and then calculate the average value of all client data Next, the server can average the data from all clients. And the total number of data from all clients Sent to each client.
[0129] In a specific implementation, in the above multi-client data processing method provided by the embodiment of the present invention, when executing step S101, each client may calculate the local partial variance using the following formula:
[0130] ; (5)
[0131] ; (6)
[0132] in, For the The local partial variance of the client, For the The local partial variance of the client Quantity, , For the Client data In line 1 The value of the column, For the Client data In the Line The value of the column, The average value of the data No. A quantity.
[0133] In implementation, if , that is No. The component is Then the local partial variance is The calculation of the components can be done according to the above formula (5), and then the The local partial variance of the client Send to the server.
[0134] In a specific implementation, in the multi-client data processing method provided in the embodiment of the present invention, when executing step S101, the server may calculate the data standard deviation using the following formula:
[0135] ; (7)
[0136] ; (8)
[0137] in, is the data standard deviation, The data variance is obtained by adding the local partial variances of all clients. is the data variance No. A quantity.
[0138] In implementation, the server can first add up the local partial variances of all clients to obtain the data variance using the above formula (7); if ,Right now No. The component is , then the above formula (8) is inferred. By taking the square root of each component of the data variance through the above formula (8), the data standard deviation can be obtained , and send it to the client.
[0139] In specific implementation, in the above multi-client data processing method provided by the embodiment of the present invention, when executing step S101, each client standardizes its own data using the following formula according to the data standard deviation:
[0140] ; (9)
[0141] ; (10)
[0142] ; (11)
[0143] in, For the Client data In the Line The data obtained after the column values are standardized. For the Client data In the Line The value of the column, is the data standard deviation No. Quantity, For the After client standardization Items of data, For the The data is standardized by the client.
[0144] In implementation, if ,Right now No. The component is . Then Each client can standardize the data according to the above formula (9).
[0145] Furthermore, in a specific implementation, in the above-mentioned multi-client data processing method provided in an embodiment of the present invention, step S101 uses a principal component analysis method to transform the standardized data, which may specifically include: calculating a local covariance matrix based on the total number of data and the standardized data and sending it to the server, so that the server calculates the overall covariance matrix, performs singular value decomposition on the overall covariance matrix, selects several eigenvectors corresponding to the largest eigenvalues and sends them to each client; each client transforms the standardized data according to the selected eigenvectors.
[0146] In the implementation, the server has sent the total number of data of all clients to each client. Each client can calculate the local covariance matrix based on the total number of data and the standardized data and send it to the server; the server can calculate the overall covariance matrix based on the local covariance matrix, perform singular value decomposition (SVD) on the overall covariance matrix, select the eigenvectors corresponding to several largest eigenvalues and send them to each client; at this time, each client can transform the standardized data according to the selected eigenvectors.
[0147] In a specific implementation, in the multi-client data processing method provided in the embodiment of the present invention, when executing step S101, the server may calculate the local covariance matrix using the following formula:
[0148] ; (12)
[0149] in, is the local covariance matrix, is the total number of data of all clients, For the After client standardization Items of data, , For the The number of data of each client. For the The standardized data of each client, Indicates transpose.
[0150] In implementation, the local covariance matrix It can be solved according to the above formula (12), and then the local covariance matrix Send to the server.
[0151] In the specific implementation, in the above-mentioned multi-client data processing method provided by the embodiment of the present invention, when executing step S101, the server calculates the overall covariance matrix, performs singular value decomposition on the overall covariance matrix, selects several eigenvectors corresponding to the largest eigenvalues and sends them to each client, which may specifically include: the server calculates the overall covariance matrix ; Then, the population covariance matrix Perform singular value decomposition and get ,in is a diagonal matrix that satisfies , is an orthogonal matrix, that is ; Finally, select Before OK Sent to each client.
[0152] In specific implementation, in the above multi-client data processing method provided by the embodiment of the present invention, when executing step S101, each client uses the following formula to transform the standardized data according to the selected feature vector:
[0153] ; (13)
[0154] ; (14)
[0155] in, For the The client is normalized to the The data obtained after the data is transformed. For the The data obtained after a client transforms the standardized data.
[0156] In implementation, Each client can transform the data according to the above formula (13). Assume that after all the data are transformed, Client data Corresponding to the above formula (14), where That is, after the transformation, the data becomes Wei.
[0157] Further, in the specific implementation, in the above-mentioned multi-client data processing method provided by the embodiment of the present invention, step S102 each client sorts each component in the transformed data, and extracts a set number of values from each sorted component according to a set interval, which may specifically include: The client will transform the processed data Quantity Sort; extract the values, among which , , To set the quantity;
[0158] If the extracted value is , No. All values extracted by the client are:
[0159] ; (15)
[0160] in, For the All values extracted by the client, is the dimension of all extracted values, , , is the dimension of all data.
[0161] In implementation, The transformed data of the client The components are sorted and then the first elements, so all the element values will be extracted Send to the server.
[0162] Furthermore, in the specific implementation, in the multi-client data processing method provided in the embodiment of the present invention, step S103 uses the extracted values, combined with the Wasserstein distance and the Mahalanobis distance to cluster all clients, which may specifically include the following steps:
[0163] Step 1: Select data from multiple clients as cluster centers;
[0164] Step 2: Calculate the target distance between each client data and each cluster center by combining Wasserstein distance with Mahalanobis distance;
[0165] Step 3: Assign each client to the class with the cluster center closest to the target;
[0166] Step 4: Update the cluster center point;
[0167] Step 5: Repeat steps 2, 3, and 4 until the cluster center point no longer changes.
[0168] In specific implementation, in the multi-client data processing method provided in the embodiment of the present invention, in the above step 1, selecting data of multiple clients as cluster centers may specifically include:
[0169] Select The data of each client is used as the cluster center;
[0170] Will The data of cluster centers are recorded as :
[0171] ; (16)
[0172] in, , To set the quantity, , is the dimension of all extracted values, , , is the dimension of all data, is the cluster center data Elements of Indicates transpose.
[0173] In a specific implementation, in the multi-client data processing method provided in the embodiment of the present invention, in the above step 2, the server may use the following formula to calculate the target distance between each client data and each cluster center obtained by combining the Wasserstein distance with the Mahalanobis distance:
[0174] ; (17)
[0175] in, For the Client data and The target distance between cluster centers, It is client data Elements of is a diagonal matrix No. diagonal elements.
[0176] The derivation process of the above formula (17) corresponding to the Wasserstein-Mahalanobis distance and the corresponding principle of the multi-client data processing method of the present invention are described below:
[0177] First, the Wasserstein distance is introduced. Consider two Polish probability spaces and and a lower semi-continuous cost function .in, and are the sample spaces of two probability distributions, and is the distribution function. The first-order Wasserstein distance is defined as:
[0178] ; (18)
[0179] in, is the first-order Wasserstein distance, yes The joint distribution of is, and its marginal distributions are and . The definition of the order Wasserstein distance is as follows:
[0180] ; (19)
[0181] for A continuous one-dimensional distribution, express The Cumulative Distribution Function (CDF) is:
[0182] ; (20)
[0183] The inverse of the CDF is its quantile function yes:
[0184] ;(twenty one)
[0185] Among them, inf represents the infimum, Represents quantile.
[0186] Similarly, for ,use Represents the corresponding function. If the cost function is .So Between The order Wasserstein distance is:
[0187] ;(twenty two)
[0188] for The approximate calculation of can be done from the probability space as follows Randomly select Data , and then arrange these data in order from small to large to get .
[0189] According to the following method Make an approximation:
[0190] ;(twenty three)
[0191] so The approximate method is:
[0192] ;(twenty four)
[0193] Similarly, from the probability space Randomly select Data , and then arrange these data in order from small to large to get , we can get The approximate value of is:
[0194] ; (25)
[0195] Among them, the present invention can take , that is:
[0196] ; (26)
[0197] If the correlation between multidimensional data is not considered, the Wasserstein distance of each dimension can be calculated separately, and then the weighted average of them can be used to obtain the overall Wasserstein distance. However, in fact, there is often a certain correlation between multidimensional data. When there is correlation between multidimensional data, Mahalanobis distance is an effective way to calculate the distance between two unknown The distance method.
[0198] Assume that the multidimensional data is:
[0199] ; (27)
[0200] For convenience, we use The form of No. The rows are converted into column vectors. .so It can also be abbreviated as follows:
[0201] ; (28)
[0202] For multidimensional data Standardize and get data ,in The covariance matrix between different dimensions after standardization can be calculated as follows:
[0203] ; (29)
[0204] data and The Mahalanobis distance between them is:
[0205] ; (30)
[0206] Assume that Perform SVD to get ,in is a diagonal matrix that satisfies , is an orthogonal matrix, that is . Then we can get:
[0207] ; (31)
[0208] If the order , we can get:
[0209] ; (32)
[0210] This is equivalent to the column vector corresponding to each row of data Make a transformation , so the covariance matrix of the transformed data is:
[0211] ; (33)
[0212] when is full rank (if , this condition is easily satisfied), It is a diagonal matrix, and the elements on the diagonal are greater than 0, which is a good property. In fact, the transformation process is basically the same as the principal component analysis process, the difference is that the principal component analysis does not select Transform the data by using all columns of .
[0213] Based on this, the present invention uses the transformed data and the corresponding Mahalanobis distance, and combines it with the Wasserstein distance to form the Wasserstein-Mahalanobis distance. The specific contents are as follows:
[0214] Assume The data of the client is ,in , that is, the dimension of all data is After standardizing the data, we get the following data: ;in The standardized covariance matrix of multidimensional data is .right Perform singular value decomposition and get . Assume that all data After the transformation The data of the client is ,in , the corresponding covariance matrix becomes If The first coordinates, sorted by size, and selected at equal intervals The data are , the same is true for The first coordinates, sorted by size, and selected at equal intervals The data are , the distance between two client data distributions is measured as follows:
[0215] ; (34)
[0216] in, , .
[0217] Simplifying it gives:
[0218] ; (35)
[0219] in, yes No. This is equivalent to finding the Wasserstein distance for each dimension of the transformed data, and then finding the square root of the linear combination of the squares of these Wasserstein distances, where the linear combination The coefficient corresponding to the term is .
[0220] If only the principal component is used for calculation, it is assumed that the transformed The first dimension plays a major role in the difference between data, while the following dimensions have almost no effect on the difference between data. Then the distance between the data distributions of two clients is calculated as follows.
[0221] The first is the data transformation process, only taking Before Column, that is Before Line, record as . Transform the data . Assume that after all data are transformed The data of the client is ,in, ;
[0222] That is, after the transformation, the data becomes Then sort the transformed data in each dimension and select them at equal intervals. number, get data ,in, ;
[0223] The following formula is obtained:
[0224] . (36)
[0225] Calculate the distance between two client data distributions. yes No. diagonal elements, and .
[0226] One advantage of this is that when the covariance matrix is not full rank, calculations can still be performed. In addition, there is another advantage of using only principal components. Since the transformed data is used for clustering calculations, the transformed data needs to be sent out. If all components are used, after others obtain the data, they can combine the orthogonal matrix The original data can be easily restored. However, others cannot restore the original data if only the principal component is used.
[0227] The present invention defines the above distance as Wasserstein-Mahalanobis distance, which satisfies non-negativity, symmetry and triangle inequality.
[0228] In implementation, when calculating the Wasserstein-Mahalanobis distance between each client data and each cluster center, the above formula (17) can be used to calculate the Client data and The target distance between cluster centers is calculated, and each client is then assigned to the class of the cluster center that is closest to it.
[0229] In specific implementation, in the above-mentioned multi-client data processing method provided in an embodiment of the present invention, in the above-mentioned step four, updating the cluster center point may specifically include: obtaining the average value of the client data belonging to each category; and using the average value of the client data belonging to each category as the new cluster center point.
[0230] In implementation, for each class of client data Find the average value, and the new cluster center is the average value.
[0231] In a specific implementation, in the multi-client data processing method provided in the embodiment of the present invention, the following formula may be used to obtain the average value of the client data belonging to each category:
[0232] ; (37)
[0233] in, For the The client data of the class, For the A collection of client indexes of the class, Representing a collection The number of elements, For the The average value of client data for the class.
[0234] In implementation, it is assumed that The collection of client indexes for a class is , then The center point of the class is updated to ,in Indicates the number of elements in this set. Elements in .
[0235] Furthermore, in specific implementation, in the above-mentioned multi-client data processing method provided in an embodiment of the present invention, in step S104, clients in different categories perform federated learning separately, which may specifically include: each client receives a global model of the corresponding category established by the server; the global model received by all clients in each category is the same; each client trains its own global model, and uploads weights and gradient parameters to the server, so that the server aggregates the model parameters in the corresponding category to obtain a new global model of the corresponding category; until the loss function of the corresponding category converges or the number of iterations reaches the set number, the trained global model of the corresponding category is output; and the trained global model of the corresponding category is used to classify or detect the task to be processed.
[0236] In implementation, since the distance between the client data distributions in each category is relatively close, it can be considered that their data is independent and identically distributed, so federated learning can be performed directly. The following operations can be performed for each category of clients: First, the server establishes a global model for the category and sends it to each client in the category. Then, the model is trained on these clients (normalized or transformed data can be used). After the training is completed, these clients upload the weights and gradient parameters to the server. Next, the server aggregates the model parameters in the category and calculates the global model of the category. The above training process and aggregation calculation steps are repeated until the loss function of the category converges or the number of iterations reaches the requirement, and the global model of the category (i.e., the target task model) is output. The model can be used to classify or personalize the tasks to be processed.
[0237] It should be pointed out that in order to handle federated learning when the client data is not independent and identically distributed data, the present invention uses Wasserstein-Mahalanobis distance to directly measure the similarity between the data distributions of two clients. Compared with other indirect methods for judging the similarity between the distributions of two clients, the present invention can use Wasserstein distance combined with Mahalanobis distance to quantify the similarity between the data distributions of two clients, and gives an analytical expression, thereby providing a judgment standard for the similarity of data distribution. Then use this distance to judge the relationship between the user data distributions, and then cluster multiple clients. After clustering is completed, the data of the clients in each category can be considered to be independent and identically distributed, so it will be more efficient to conduct federated learning in each category, and this federated learning can perform some personalized predictions.
[0238] In the above embodiments, the multi-client data processing method is described in detail, and the present invention also provides corresponding embodiments of the multi-client data processing system and the multi-client data processing device. It should be noted that the present invention describes the embodiments of the system part from two perspectives, one is based on the perspective of functional modules, and the other is based on the perspective of hardware.
[0239] Based on the perspective of functional modules, this embodiment provides a multi-client data processing system, which includes multiple clients and at least one server;
[0240] Each client is used to standardize its own data and transform the standardized data using the principal component analysis method; it is also used to sort each component in the transformed data, and extract a set number of values from each sorted component at a set interval, and send the extracted values to the server;
[0241] The server is used to cluster all clients using the extracted values combined with Wasserstein distance and Mahalanobis distance;
[0242] Clients in different categories are used to perform federated learning separately.
[0243] In the multi-client data processing system provided in the embodiment of the present invention, calculation errors can be reduced, the dimension of the data can be reduced, the leakage of the original data can be avoided, and the Wasserstein distance is combined with the Mahalanobis distance to directly quantify the similarity between the distributions of multi-client data. After all clients are clustered, the data of the clients in each category can be considered to be independent and identically distributed. Therefore, it is more efficient to directly perform federated learning in each category, and the federated learning can perform some personalized predictions, thereby improving the system's processing efficiency for non-independent and identically distributed data.
[0244] Since the embodiments of the system part and the embodiments of the method part correspond to each other, the embodiments of the system part refer to the description of the embodiments of the method part, which will not be repeated here. And it has the same beneficial effects as the multi-client data processing method mentioned above.
[0245] Furthermore, in a specific implementation, in the multi-client data processing system provided in the embodiment of the present invention, each client can be used to obtain the sum of all its own data; send the obtained sum of all its own data and the number of its own data to the server;
[0246] The server can be used to calculate the average data value of all clients and send the average data value to each client;
[0247] Each client can be used to calculate the local partial variance based on the data average and send it to the server;
[0248] The server can be used to calculate the standard deviation of the data and send it to each client;
[0249] Each client can specifically be used to standardize its own data according to the data standard deviation.
[0250] In implementation, each client can, after obtaining the sum of all its own data, send the sum of all its own data and the number of its own data to the server; after receiving the sum of all its own data and the number of its own data obtained by each client, the server can calculate the data average of all clients and send the data average to each client; at this time, each client can calculate the local partial variance based on the data average and send it to the server; after receiving the local partial variance, the server can calculate the data standard deviation based on the local partial variance and send it to each client; after receiving the data standard deviation, each client can standardize its own data based on the data standard deviation.
[0251] Furthermore, in a specific implementation, in the multi-client data processing system provided in the embodiment of the present invention, each client can be used to calculate a local covariance matrix based on the total number of data and the standardized data and send it to the server;
[0252] The server can be used to calculate the overall covariance matrix, perform singular value decomposition on the overall covariance matrix, select several eigenvectors corresponding to the largest eigenvalues and send them to each client;
[0253] Each client may be specifically used to perform transformation processing on the standardized data according to the selected feature vector.
[0254] In implementation, after each client receives the total number of data and the standardized data sent by the server, it can calculate the local covariance matrix based on the total number of data and the standardized data and send it to the server; after receiving the local covariance matrix, the server can first calculate the overall covariance matrix based on the local covariance matrix, and then perform singular value decomposition on the overall covariance matrix, select several eigenvectors corresponding to the largest eigenvalues and send them to each client; at this time, each client can transform the standardized data according to the selected eigenvectors.
[0255] Furthermore, in a specific implementation, in the multi-client data processing system provided in an embodiment of the present invention, the server can be specifically used to select data of multiple clients as cluster centers; calculate the target distance between each client data and each cluster center obtained by combining the Wasserstein distance with the Mahalanobis distance; classify each client into the class where the cluster center closest to the target is located; update the cluster center point; repeat the above-mentioned target distance calculation steps, classify each client into the class where the cluster center closest to the target is located and update the cluster center point, until the cluster center point no longer changes.
[0256] In implementation, the server may first select The data of each client is used as the cluster center; The data of cluster centers are recorded as ; Then, calculate the target distance between each client data and each cluster center by combining Wasserstein distance and Mahalanobis distance; then assign each client to the class where the cluster center closest to it is located. Then, obtain the average value of the client data belonging to each class; take the average value of the client data belonging to each class as the new cluster center point. Repeat the above target distance calculation steps, assign each client to the class where the cluster center closest to the target distance is located and update the cluster center point until the cluster center point no longer changes.
[0257] Furthermore, in the specific implementation, in the above-mentioned multi-client data processing method provided by the embodiment of the present invention, each client can be specifically used to receive the global model of the corresponding category established by the server; all clients in each category receive the same global model; train their own global model, and upload the weights and gradient parameters to the server;
[0258] The server may be specifically used to aggregate the model parameters in the corresponding category to obtain a new global model of the corresponding category, and send the new global model of the corresponding category to the client of the corresponding category;
[0259] The client of the corresponding category can be used to continue training its own global model until the loss function of the corresponding category converges or the number of iterations reaches a set number, and output the trained global model of the corresponding category to use the trained global model of the corresponding category to classify or detect the task to be processed.
[0260] In implementation, since the distance between the client data distributions in each category is relatively close, it can be considered that their data is independent and identically distributed, so federated learning can be performed directly in each category. First, the server establishes a global model for the category and sends it to each client in the category. Then, the global model is trained in these clients in the category. After the training is completed, these clients upload the weights and gradient parameters to the server. Next, the server aggregates the model parameters in the category and calculates the global model of the category. The above training process and aggregation calculation steps are repeated until the loss function of the category converges or the number of iterations reaches the requirement, and the target task model corresponding to the category is output.
[0261] Figure 2 This is a structural diagram of a multi-client data processing device provided by an embodiment of the present invention. This embodiment is based on the hardware perspective, such as Figure 2As shown, the multi-client data processing device includes:
[0262] A memory 20, for storing computer programs;
[0263] The processor 21 is used to implement the steps of the multi-client data processing method mentioned in the above embodiment when executing the computer program.
[0264] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU; the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.
[0265] The memory 20 may include one or more non-volatile storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the multi-client data processing method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. Data 203 may include, but is not limited to, the data involved in the multi-client data processing method mentioned above.
[0266] In some embodiments, the multi-client data processing device may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26. Those skilled in the art will appreciate that Figure 2The structure shown in does not constitute a limitation on the multi-client data processing device, and may include more or fewer components than shown in the figure. The multi-client data processing device provided by the embodiment of the present invention includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the following method: a multi-client data processing method, the effect is the same as above.
[0267] Finally, the present invention also provides an embodiment corresponding to a non-volatile storage medium. The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps recorded in the above method embodiment are implemented.
[0268] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code. The non-volatile storage medium provided by the present invention can implement the multi-client data processing method mentioned above, and the effect is the same as above.
[0269] Finally, the present invention also provides an embodiment corresponding to a computer program product. The computer program product includes a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps recorded in the above multi-client data processing method embodiment are implemented. The computer program product provided by the present invention can implement the multi-client data processing method mentioned above, and the effect is the same as above.
[0270] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.
[0271] The multi-client data processing method, system, device, medium and product provided by the present invention are introduced in detail above. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.
Claims
1. A multi-client data processing method, characterized in that: The method comprises: Each client standardizes its own data and uses the principal component analysis method to transform the standardized data; Each client sorts each component in the transformed data, and extracts a set number of values from each sorted component at a set interval, and uses the extracted values to approximate the distribution obeyed by the client data; Each client sends the extracted value to the server, so that the server uses the extracted value, combined with Wasserstein distance and Mahalanobis distance to cluster all clients, and directly measure the similarity between multi-client data distributions, including: The data of each client is used as the cluster center; The data of cluster centers are recorded as : ; in, , For the set quantity, , is the dimension of all extracted values, , , is the dimension of all data, is the cluster center data Elements of Represents transposition; the following formula is used to calculate the target distance between each client data and each cluster center, which is obtained by combining the Wasserstein distance with the Mahalanobis distance: ; in, For the Client data and The target distance between cluster centers, It is client data Elements of is a diagonal matrix No. diagonal elements; assign each client to the class where the cluster center closest to the target is located; update the cluster center point; repeat the steps of calculating the target distance and updating the cluster center point until the cluster center point no longer changes; client data For the All values extracted by the client: ; Diagonal Matrix It is obtained by singular value decomposition of the overall covariance matrix, which is obtained by summing the local covariance matrices calculated from the total number of data and the standardized data; After all clients are clustered, clients in different categories perform federated learning separately.
2. The multi-client data processing method according to claim 1, characterized in that: Each client standardizes its own data, including: Each client obtains the sum of all its own data; Each client sends the sum of all its own data and the number of its own data to the server, so that the server calculates the average value of all the clients' data and sends the average value of the data to each client; Each client calculates a local partial variance based on the data average value and sends the variance to the server, so that the server calculates the data standard deviation and sends the variance to each client; Each client standardizes its own data according to the data standard deviation.
3. The multi-client data processing method according to claim 2, characterized in that: Each client uses the following formula to obtain the sum of all its data: ; in, For the The sum of all data of the clients, The vector is represented in the form of The form of Client data No. The rows are converted into column vectors. Indicates Client data No. The value of row 1, column 1, Indicates Client data No. Line The value of the column, , For the The number of data of each client. is the dimension of all data, represents the summation symbol, Indicates transpose.
4. The multi-client data processing method according to claim 3, characterized in that: The following formula is used to calculate the average value of data for all clients: ; ; ; in, Represents the total data of all clients. , is the total number of clients, is the number of data of the first client, For the The number of data of each client. is the total number of data of all clients, It is the average value of data of all clients.
5. The multi-client data processing method according to claim 4, characterized in that: Each client calculates the local partial variance using the following formula: ; ; in, For the The local partial variance of the client, For the The local partial variance of the client Quantity, , For the Client data In line 1 The value of the column, For the Client data In the Line The value of the column, The average value of the data No. A quantity.
6. The multi-client data processing method according to claim 5, characterized in that: The standard deviation of the data is calculated using the following formula: ; ; in, is the standard deviation of the data, The data variance is obtained by adding the local partial variances of all clients. is the data variance No. A quantity.
7. The multi-client data processing method according to claim 6, characterized in that: Each client standardizes its own data using the following formula based on the data standard deviation: ; ; ; in, For the Client data In the Line The data obtained after the column values are standardized. For the Client data In the Line The value of the column, is the standard deviation of the data No. Quantity, For the After client standardization Items of data, For the The data is standardized by each client.
8. The multi-client data processing method according to claim 1, characterized in that: The principal component analysis method is used to transform the standardized data, including: The total number of data sent by the receiving server; According to the total number of data and the standardized data, a local covariance matrix is calculated and sent to the server, so that the server calculates the overall covariance matrix, performs singular value decomposition on the overall covariance matrix, selects several eigenvectors corresponding to the largest eigenvalues and sends them to each client; Each client performs transformation processing on the standardized data according to the selected feature vector.
9. The multi-client data processing method according to claim 8, characterized in that: The local covariance matrix is calculated using the following formula: ; in, is the local covariance matrix, is the total number of data of all clients, For the After client standardization Items of data, , For the The number of data of each client. For the The standardized data of each client, Indicates transpose.
10. The multi-client data processing method according to claim 9, characterized in that: The server calculates the overall covariance matrix, performs singular value decomposition on the overall covariance matrix, selects several eigenvectors corresponding to the largest eigenvalues and sends them to each client, including: The server calculates the overall covariance matrix ; For the population covariance matrix Perform singular value decomposition and get ,in is a diagonal matrix that satisfies , is an orthogonal matrix; Select Before OK Sent to each client.
11. The multi-client data processing method according to claim 10, characterized in that: Each client uses the following formula to transform the standardized data according to the selected feature vector: ; ; in, For the The client is normalized to the The data obtained after the data is transformed. For the The data obtained after a client transforms the standardized data.
12. The multi-client data processing method according to claim 11, characterized in that: Each client sorts each component in the transformed data, and extracts a set number of values from each sorted component at a set interval, including: No. The client will transform the processed data Quantity Sort by Extract the values, among which , , The set quantity is as described; If the extracted value is , No. All values extracted by the client are .
13. The multi-client data processing method according to claim 1, characterized in that: Update cluster centers, including: Get the average value of client data belonging to each category; The average value of the client data belonging to each category is used as the new cluster center point.
14. The multi-client data processing method according to claim 13, characterized in that: The following formula is used to obtain the average value of client data belonging to each category: ; in, For the The client data of the class, For the A collection of client indexes of the class, Representing a collection The number of elements, For the The average value of client data for the class.
15. The multi-client data processing method according to claim 14, characterized in that: Clients in different categories perform federated learning separately, including: Each client receives a global model of the corresponding category established by the server; all clients in each category receive the same global model; Each client trains its own global model and uploads the weights and gradient parameters to the server, so that the server aggregates the model parameters within the corresponding category to obtain a new global model of the corresponding category; Until the loss function of the corresponding category converges or the number of iterations reaches the set number, the trained global model of the corresponding category is output; Use the trained global model of the corresponding category to classify or detect the task to be processed.
16. A multi-client data processing system, characterized in that: The system includes a plurality of clients and at least one server; Each client is used to standardize its own data and transform the standardized data using the principal component analysis method; it is also used to sort each component in the transformed data, and extract a set number of values from each sorted component at a set interval, use the extracted values to approximate the distribution obeyed by the client data, and send the extracted values to the server; The server is used to cluster all clients using the extracted values, combined with Wasserstein distance and Mahalanobis distance, to directly measure the similarity between multi-client data distributions, including: The data of each client is used as the cluster center; The data of cluster centers are recorded as : ; in, , For the set quantity, , is the dimension of all extracted values, , , is the dimension of all data, is the cluster center data Elements of Represents transposition; the following formula is used to calculate the target distance between each client data and each cluster center, which is obtained by combining the Wasserstein distance with the Mahalanobis distance: ; in, For the Client data and The target distance between cluster centers, It is client data Elements of is a diagonal matrix No. diagonal elements; assign each client to the class where the cluster center closest to the target is located; update the cluster center point; repeat the steps of calculating the target distance and updating the cluster center point until the cluster center point no longer changes; client data For the All values extracted by the client: ; Diagonal Matrix It is obtained by singular value decomposition of the overall covariance matrix, which is obtained by summing the local covariance matrices calculated from the total number of data and the standardized data; Clients in different categories are used to perform federated learning separately.
17. A multi-client data processing device, characterized in that: The device comprises: Memory for storing computer programs; A processor, configured to implement the steps of the multi-client data processing method as described in any one of claims 1 to 15 when executing the computer program.
18. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the multi-client data processing method according to any one of claims 1 to 15 are implemented.
19. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the multi-client data processing method as described in any one of claims 1 to 15 are implemented.
