Data processing method and device for federated learning model, medium and program product

Abstract

This application discloses a data processing method, apparatus, medium, and program product for a federated learning model. The method includes: obtaining a feature set sent by the participants of the federated learning model; for each feature subset, calculating the standard deviation of the samples corresponding to each feature subset based on the feature vectors; calculating a set of discrete deviations based on the feature subsets and their corresponding sample standard deviations, the set of discrete deviations including discrete deviations corresponding one-to-one with the feature vectors of each feature subset; performing clustering processing on the set of discrete deviations, and determining the outliers in the set of discrete deviations based on the clustering results. According to embodiments of this application, outlier feature vectors can be identified, thereby enabling processing of outlier feature vectors to improve the accuracy of feature vector aggregation calculations.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to a data processing method, apparatus, medium and program product for a federated learning model. Background Technology

[0002] In existing horizontal federated learning model training processes, each participant calculates intermediate feature vectors based on its own data and sends these vectors to the model aggregation node. The aggregation node then weights the feature vectors based on the amount of data from each participant and feeds the results back to each participant for continued federated learning model training. Alternatively, the aggregation node can further train the federated learning model based on its own results. When a participant with a large amount of data contains a significant amount of noisy data, the weight corresponding to that participant will be substantial, greatly affecting the accuracy of the feature vector aggregation calculation and consequently impacting the accuracy of the trained federated learning model. Summary of the Invention

[0003] This application provides a data processing method, apparatus, medium, and program product for a federated learning model, which can solve the problem that in the training process of existing federated learning models, when a participant has a large amount of noisy data, the accuracy of feature vector aggregation calculation is low.

[0004] In a first aspect, embodiments of this application provide a data processing method for a federated learning model, the method comprising:

[0005] Obtain the feature set sent by the participants of the federated learning model. The feature set includes at least one feature subset, and the feature subset includes multiple feature vectors.

[0006] For each feature subset, calculate the standard deviation of the samples corresponding to each feature subset based on the feature vector;

[0007] Based on the feature subsets and the standard deviation of the samples corresponding to them, calculate the discrete deviation set corresponding to each feature subset. The discrete deviation set includes the discrete deviation that corresponds one-to-one with each feature vector.

[0008] Cluster the discrete deviation set, determine the abnormal deviation in the discrete deviation set based on the clustering results, and identify the feature vectors corresponding to the abnormal deviations.

[0009] In some embodiments, the discrete deviation set is clustered, and the abnormal deviations in the discrete deviation set are determined based on the clustering results. The feature vectors corresponding to the abnormal deviations are then identified, including:

[0010] For each discrete deviation set corresponding to a feature subset, a coverage area with each discrete deviation as the cluster center is determined.

[0011] For each coverage area, the aggregation factor corresponding to the coverage area is calculated based on the similarity between each discrete offset and the cluster center;

[0012] Based on the aggregation factor corresponding to each coverage area, the abnormal deviation degree in the discrete deviation degree set is determined, and the feature vector corresponding to the abnormal deviation degree is identified.

[0013] In some embodiments, for each discrete deviation set corresponding to a feature subset, a coverage area is determined with each discrete deviation as the cluster center, including:

[0014] For each discrete offset set corresponding to a feature subset, each discrete offset is set as a cluster center, and the Euclidean distance between the cluster center and other discrete offsets is sorted.

[0015] Set the Euclidean distance in the preset order to the domain radius corresponding to the cluster center;

[0016] Set the set of discrete offsets whose Euclidean distance is less than the domain radius as the coverage area corresponding to the cluster center.

[0017] In some embodiments, for each coverage area, a aggregation factor corresponding to the coverage area is calculated based on the similarity between each discrete offset and the cluster centers, including:

[0018] For each coverage area, the average discrete distance to the coverage area is calculated based on the similarity between each discrete offset in the coverage area and the cluster center.

[0019] Based on the average discrete distance, the discrete density corresponding to each cluster center is calculated, and the discrete density corresponds one-to-one with the discrete offset.

[0020] For each coverage area, calculate the density ratio of other discrete offsets in the coverage area to which the cluster center belongs to the cluster center, and calculate the average density ratio based on the density ratio. Set the average density ratio as the aggregation factor corresponding to the cluster center.

[0021] In some embodiments, for each coverage area, the average discrete distance corresponding to the coverage area is calculated based on the similarity between each discrete offset in the coverage area and the cluster center, including:

[0022] For a discrete offset in each coverage area, determine the maximum value between the discrete distance between the cluster center and the discrete offset, and the Euclidean distance between the cluster center and the discrete offset, and set the maximum value as the similarity between the cluster center and the discrete offset.

[0023] The average discrete distance corresponding to the coverage area is calculated based on the similarity between the cluster centers and discrete offsets in the coverage area.

[0024] In some embodiments, for each feature subset, the standard deviation of the samples corresponding to each feature subset is calculated based on the feature vector, including:

[0025] For each feature subset, calculate the feature mean corresponding to the feature subset based on the feature vector;

[0026] Calculate the sample standard deviation of each feature subset based on the feature vector and feature mean corresponding to each feature subset.

[0027] In some embodiments, a set of discrete deviations is calculated based on a subset of features and the standard deviation of the samples corresponding to it, including:

[0028] Based on the eigenvectors, eigenmeans, and sample standard deviations corresponding to each feature subset, calculate the discrete offset corresponding to each feature vector.

[0029] Among them, the discrete offsets corresponding to the feature vectors in the same feature subset form a discrete deviation set.

[0030] In some embodiments, the method further includes:

[0031] Delete the feature vectors corresponding to the abnormal deviations from the feature set, and delete the abnormal deviations from the discrete deviation set.

[0032] The weights corresponding to each feature vector are calculated based on the set of discrete deviations, and the feature vectors are corrected based on the weights.

[0033] In some embodiments, calculating the weights corresponding to each feature vector based on the set of discrete deviations, and correcting the feature vectors based on the weights, includes:

[0034] For the discrete deviation degree corresponding to a feature subset, normalization is performed, and a judgment matrix is ​​constructed based on the ratio between each pair of the normalized discrete deviation degrees.

[0035] Based on the judgment matrix, calculate the weights corresponding to the feature subsets and participants, and correct the feature vectors according to the weights.

[0036] Secondly, embodiments of this application provide a data processing apparatus, the apparatus comprising:

[0037] The acquisition module is used to acquire the feature set sent by the participants of the federated learning model. The feature set includes at least one feature subset, and the feature subset includes multiple feature vectors.

[0038] The calculation module is used to calculate the standard deviation of the samples corresponding to each feature subset based on the feature vector.

[0039] The calculation module is also used to calculate the discrete deviation set corresponding to each feature subset based on the feature subset and the standard deviation of the sample corresponding to it. The discrete deviation set includes the discrete deviation that corresponds one-to-one with each feature vector.

[0040] The identification module is used to perform clustering on the discrete deviation set, determine the abnormal deviation in the discrete deviation set based on the clustering results, and identify the feature vector corresponding to the abnormal deviation.

[0041] Thirdly, embodiments of this application provide a data processing device, the device including: a processor and a memory storing computer program instructions;

[0042] The data processing method of the federated learning model described above is implemented when the processor executes computer program instructions.

[0043] Fourthly, embodiments of this application provide a computer storage medium storing computer program instructions, which, when executed by a processor, implement the data processing method of the federated learning model described above.

[0044] Fifthly, embodiments of this application provide a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, they implement the data processing method of the federated learning model described above.

[0045] The data processing method, apparatus, medium, and program product of the federated learning model provided in this application calculate the sample standard deviation for each feature subset, and calculate the discrete deviation degree corresponding one-to-one with the feature vector of each feature subset through the sample standard deviation. Thus, the discrete deviation degree can reflect the degree of dispersion of each feature vector. Furthermore, by performing clustering processing on the discrete deviation degree set, and determining the abnormal deviation degree in the discrete deviation degree set according to the clustering processing result, the feature vector corresponding to the abnormal deviation degree can be identified. Further processing such as removing, correcting, and reducing the weight of the feature vector corresponding to the abnormal deviation degree can be performed to improve the accuracy of feature vector aggregation calculation. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1This is a flowchart illustrating the data processing method of a federated learning model provided in an embodiment of this application;

[0048] Figure 2 This is a detailed flowchart of S200 and S300 in the data processing method of the federated learning model provided in an embodiment of this application;

[0049] Figure 3 This is a detailed flowchart of S400 in the data processing method of the federated learning model provided in an embodiment of this application;

[0050] Figure 4 This is a detailed flowchart of S410 in the data processing method of the federated learning model provided in an embodiment of this application;

[0051] Figure 5 This is a schematic diagram of the coverage area provided in one embodiment of this application;

[0052] Figure 6 This is a detailed flowchart of S420 in the data processing method of the federated learning model provided in an embodiment of this application;

[0053] Figure 7 This is a partial flowchart of the data processing method for a federated learning model provided in another embodiment of this application;

[0054] Figure 8 This is a detailed flowchart of S600 in the data processing method of the federated learning model provided in an embodiment of this application;

[0055] Figure 9 This is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of this application;

[0056] Figure 10 This is a schematic diagram of the hardware structure of a data processing device provided in an embodiment of this application. Detailed Implementation

[0057] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples of this application.

[0058] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0059] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The embodiments will now be described in detail with reference to the accompanying drawings.

[0060] In the training process of existing horizontal federated learning models, each participant calculates the intermediate process feature vector based on its own data and sends the feature vector to the model aggregation node. The model aggregation node performs weighted calculations on the feature vectors based on the amount of data of each participant and feeds back the calculated results to each participant to continue the training of the federated learning model, or the model aggregation node further trains the federated learning model based on the calculated results.

[0061] For example, in the training process of a horizontal federated learning model at the center point of a mobile communication user activity area, assume that each participant includes a set of data p = {p1, p2, ..., p...} n}, at a certain moment, the longitudes of the center points of the activity area are b = {b1, b2, ..., b} n The formula for calculating the sum of longitudes of the center point of the activity area is as follows: in, This represents the total number of data points from all participating parties.

[0062] This weighted calculation method based on the amount of data held by each participant is relatively simple. When the data quality of each data owner is similar, this method based on data volume can yield relatively accurate summary results. However, when a participant with a large amount of data has a lot of noisy data, the weight of that participant's parameters will be very large, which will greatly affect the accuracy of parameter calculation and thus the accuracy of the trained federated learning model.

[0063] To address the problems of the prior art, embodiments of this application provide a data processing method, apparatus, medium, and program product for a federated learning model. The data processing method for the federated learning model provided in this application embodiment will be described first below.

[0064] Figure 1 A flowchart illustrating a data processing method for a federated learning model according to an embodiment of this application is shown. The method includes the following steps:

[0065] S100, Obtain the feature set sent by the participants of the federated learning model. The feature set includes at least one feature subset, and the feature subset includes multiple feature vectors.

[0066] The participants in a federated learning model are the owners of the original data used in its training. Each participant calculates parameters based on the received federated learning model sequence algorithm and their own original data, and then sends the intermediate parameters to the model aggregation node. The data processing method for the federated learning model provided in this application is applied to the model aggregation node. The model aggregation node aggregates the feature vectors sent by each participant to obtain a feature set.

[0067] Those skilled in the art will understand that participants can simultaneously send at least one type of feature vector, and the feature vectors of the same type sent by all participants are aggregated to form a feature subset. For example, if the participants are operator A, operator B, and operator C, and operator A provides a feature vector a related to user activity time. 11 ~a 1m Feature vector a related to the user's active address 21 ~a 2m Operator B provided a feature vector b related to user activity time. 11 ~b 1m Feature vector b related to user active addresses 21 ~b 2m Operator C provides a feature vector c related to user activity time. 11 ~c 1m Feature vector c related to user active addresses 21 ~c 2m Therefore, the feature set includes a subset of features related to user activity time, {a}. 11 ~a 1m b 11 ~b 1m c 11 ~c 1m} and a subset of features related to the user's active address {a 21 ~a 2m b 21 ~b2m c 21 ~c 2m Each feature subset includes 3m feature vectors.

[0068] S200, for each feature subset, calculates the standard deviation of the samples corresponding to each feature subset based on the feature vector;

[0069] Please refer to the following: Figure 2 The aforementioned S200 includes:

[0070] S210, For each feature subset, calculate the feature mean corresponding to the feature subset based on the feature vector;

[0071] S220, calculate the sample standard deviation of each feature subset based on the feature vector and feature mean corresponding to each feature subset.

[0072] The feature mean reflects the center around which the fluctuations of the feature vectors in a feature subset revolve. The sample standard deviation represents the dispersion of the feature vectors. The feature mean can be calculated using the following formula:

[0073]

[0074]

[0075] Among them, E m Let be the feature mean corresponding to the m-th feature subset, n be the total number of feature vectors in the feature subset, a be the feature vector, and σ be the feature mean. m Let be the sample standard deviation corresponding to the m-th feature subset.

[0076] For example, in one embodiment, feature vectors A, B, C, and D are included, and the feature vectors of the four feature subsets are shown in the table below:

[0077] Participants <![CDATA[Feature vector a1]]> <![CDATA[Feature vector a2]]> <![CDATA[Feature vector a3]]> <![CDATA[Feature vector a4]]> A 1.000000 4.000000 3.000000 2.000000 B 2.000000 1.000000 4.000000 3.000000 C 3.000000 2.000000 1.000000 4.000000 D 4.000000 3.000000 2.000000 1.000000

[0078] According to the aforementioned formula, σ = {1.12, 1.12, 1.12, 1.12} can be calculated.

[0079] S300, calculate the discrete deviation set corresponding to each feature subset based on the feature subset and the standard deviation of the sample corresponding to it. The discrete deviation set includes the discrete deviation that corresponds one-to-one with the feature vector of each feature subset.

[0080] The dispersion measure reflects the proportion of the absolute difference between each feature vector and the feature mean to the sample standard deviation. A larger dispersion measure indicates a more unbalanced structure; a smaller dispersion measure indicates a more balanced structure. The dispersion measure can be calculated separately for each feature vector, and then the dispersion measures can be aggregated to generate a dispersion measure set. In one embodiment, the above S300 includes:

[0081] S310, calculate the discrete offset degree corresponding to each feature vector based on the feature vector, feature mean and sample standard deviation corresponding to each feature subset, wherein the discrete offset degree corresponding to the feature vector in the same feature subset forms a discrete offset degree set.

[0082] The discrete offset can be calculated using the following formula:

[0083] Where λ is the discrete offset, a is the feature vector, E is the feature mean, and σ is the sample standard deviation.

[0084] S400 performs clustering on the discrete deviation set, determines the abnormal deviation in the discrete deviation set based on the clustering results, and identifies the feature vectors corresponding to the abnormal deviations.

[0085] One or more of the following methods can be used as needed: K-Means, mean-shift clustering, density clustering, agglomerative hierarchical clustering, etc., to cluster the discrete deviations in the discrete deviation set, thereby filtering out the abnormal discrete deviations that are far away from each cluster. If the abnormal deviations are filtered out, the feature vectors corresponding to the abnormal deviations will be significantly different from other feature vectors. Further processing such as removing, correcting, and reducing the weights of the feature vectors corresponding to the abnormal deviations can be performed to improve the accuracy of the feature vector aggregation calculation.

[0086] The data processing method, apparatus, medium, and program product of the federated learning model provided in this application calculate the sample standard deviation for each feature subset, and calculate the discrete deviation degree corresponding one-to-one with the feature vector of each feature subset through the sample standard deviation, so that the discrete deviation degree can reflect the degree of dispersion of each feature vector; further, the discrete deviation degree set is clustered, and the abnormal deviation degree in the discrete deviation degree set is determined according to the clustering result, so as to identify the feature vector corresponding to the abnormal deviation degree.

[0087] As an optional embodiment, please refer to Figure 3 The aforementioned S400 may include:

[0088] S410, For each discrete deviation set corresponding to a feature subset, determine the coverage area with each discrete deviation as the cluster center;

[0089] In S410, a preset radius for each domain and a preset number of feature vectors in each domain can be pre-set. For each feature subset corresponding to a discrete offset, a coverage area with each discrete offset as the cluster center is determined. For example, if a feature subset M includes n feature vectors, and each of the n feature vectors corresponds to one of the n discrete offsets, then feature vector a is used as the cluster center, and the set of feature vectors whose distance from feature vector a is less than or equal to a preset distance d is set as the coverage area of ​​feature vector a. Similarly, feature vector b is used as the cluster center, and the set of feature vectors whose distance from feature vector b is less than or equal to a preset distance d is set as the coverage area of ​​feature vector b. This process continues until each feature vector in the feature subset has a corresponding coverage area.

[0090] Please refer to the following: Figure 4 In one embodiment, to set a differentiated coverage area for each discrete offset, the above-described S410 includes:

[0091] S411, for each discrete deviation set corresponding to a feature subset, set each discrete deviation as a cluster center, and sort the Euclidean distances between the cluster centers and other discrete deviations;

[0092] S412, set the Euclidean distance in the preset order to the domain radius corresponding to the cluster center;

[0093] S413 sets the set of discrete offsets whose Euclidean distance is less than the domain radius as the coverage domain corresponding to the cluster center.

[0094] For each feature subset corresponding to a set of discrete offsets, the Euclidean distance between each pair of discrete offsets can be calculated. Each discrete offset is then designated as a cluster center, and the remaining discrete offsets are sorted in ascending order of their Euclidean distances to the cluster centers. The L-th discrete offset *a* closest to the current cluster center is determined. d And set the discrete offset a d The Euclidean distance to the current cluster center is the domain radius d. The set of discrete offsets whose Euclidean distance is less than the domain radius d is defined as the set of discrete offsets a. d The corresponding coverage area. That is, each coverage area contains the same number of discrete offsets, but the domain radius of the coverage area corresponding to each cluster center is not necessarily the same.

[0095] Please refer to the following: Figure 5The graph is constructed based on the Euclidean distance between two discrete offsets. If the preset order is 3, when discrete offset A is the cluster center, the corresponding coverage area includes discrete offsets A, B, C, and D; when discrete offset F is the cluster center, the corresponding coverage area includes discrete offsets C, D, E, and F. The radii of the two coverage areas are different. Those skilled in the art can set the size of the preset order as needed. It is understood that if the preset order L is too large, it will increase the inclusion of abnormal deviations in the subsequent calculation of the aggregation factor, resulting in a large aggregation factor, which may cause normal nodes to be identified as abnormal deviations. If the preset order L is too small, the aggregation factor will be very small, making it difficult to identify abnormal deviations.

[0096] By determining the Lth discrete offset a that is closest to the cluster center d And set the discrete offset a d The Euclidean distance to the current cluster center is the domain radius d. Thus, for each cluster center and other discrete offsets, a coverage domain with a different domain radius d is set.

[0097] After S410, execute: S420, for each coverage area, calculate the aggregation factor corresponding to the coverage area based on the similarity of each discrete offset and cluster center;

[0098] The aggregation factor represents the degree of aggregation for each coverage area. In a feature subset, the deviations of normal feature vectors have relatively uniform spacing and density, while the dispersion of abnormal feature vectors is far from the normal dispersion, with lower density and greater dispersion. Therefore, a higher aggregation factor indicates a greater density of feature vectors near the cluster center, and a more normal value for the cluster center; a lower aggregation factor indicates a smaller density of feature vectors near the cluster center, and a more abnormal value for the cluster center. In one embodiment, for a coverage area, similarity is the distance between each discrete offset and the cluster center, while the aggregation factor is the average distance, representing the degree of aggregation of the coverage area.

[0099] Please refer to the following: Figure 6 In another embodiment, the above-described S420 includes:

[0100] S421, For each coverage area, calculate the average discrete distance to the coverage area based on the similarity between each discrete offset in the coverage area and the cluster center;

[0101] Similarity reflects the degree of similarity between two discrete offsets. In step S421, those skilled in the art can choose an appropriate calculation method to calculate the similarity. For example, the similarity can be calculated by the Euclidean distance between the two discrete offsets, or by the numerical difference between the two discrete offsets. The average discrete distance is the sum of multiple similarities divided by the total number of similarities.

[0102] In some embodiments, S421 includes:

[0103] A. For a discrete offset in each coverage area, determine the maximum value between the discrete distance between the cluster center and the discrete offset, and the Euclidean distance between the cluster center and the discrete offset, and set the maximum value as the similarity between the cluster center and the discrete offset;

[0104] B. Calculate the average discrete distance corresponding to the coverage area based on the similarity between the cluster centers and discrete offsets in the coverage area.

[0105] This means that by combining parameters such as the feature subset identification name associated with the discrete offset, the participants sending the feature vector, etc., the Euclidean distance between the cluster center and the discrete offset can be calculated, and the numerical difference between the cluster center and the discrete offset can also be calculated as the discrete distance. The maximum value of the Euclidean distance and the discrete distance is set as the similarity between the cluster center and the discrete offset, so that the similarity can better reflect the similarity between the cluster center and the discrete offset from multiple dimensions.

[0106] For example: within the same coverage area, calculate the Euclidean distance dis between the cluster center u and the discrete offset v. l Given u and v, calculate the discrete distance d(u,v) between u and v, then the similarity is:

[0107] realdis(u,v)=max{d(u,v),dis l (u)};

[0108] The average discrete distance is:

[0109]

[0110] After S421, execute: S422, calculate the discrete density corresponding to each discrete offset as the cluster center based on the average discrete distance, and the discrete density corresponds one-to-one with the discrete offset.

[0111] S423, for each coverage area, calculate the density ratio of other discrete offsets in the coverage area to which the cluster center belongs to the cluster center, and calculate the average density ratio based on the density ratio, and set the average density ratio as the aggregation factor.

[0112] The discrete density is the reciprocal of the average discrete distance. The aggregation factor is calculated using the following formula:

[0113] density l (q) is the aggregation factor.

[0114] After step S420, execute:

[0115] S430, based on the aggregation factor corresponding to each coverage area, determine the abnormal deviation degree in the discrete deviation degree set, and identify the feature vector corresponding to the abnormal deviation degree.

[0116] Specifically, the aggregation factor is compared with a preset threshold. If the aggregation factor is greater than the preset threshold, it indicates that the value of the cluster center of the coverage area differs significantly from the overall data, and the feature vector corresponding to the cluster center is considered abnormal. If the aggregation factor is less than or equal to the preset threshold, it indicates that the value of the cluster center of the coverage area differs slightly from the overall data, and the feature vector corresponding to the cluster center is considered normal.

[0117] As an optional embodiment, please refer to Figure 7 To improve the accuracy of subsequent federated learning model parameter calculations, the S400 specification also includes:

[0118] S500: Delete the feature vector corresponding to the abnormal deviation in the feature set, and delete the abnormal deviation in the discrete deviation set.

[0119] S600 calculates the weights corresponding to each eigenvector based on the discrete deviation set, and corrects the eigenvectors based on the weights.

[0120] By directly deleting abnormal feature vectors, the impact of these abnormal feature vectors on subsequent model training is avoided. By calculating the weights corresponding to each feature vector in the domain, corrections can be made for each feature vector before model training. Those skilled in the art will understand that a larger degree of dispersion corresponds to a smaller weight, and vice versa, thus correcting the feature vectors.

[0121] In one embodiment, please refer to Figure 8 The aforementioned S600 includes:

[0122] S610: Normalize the discrete deviation degree corresponding to a feature subset, and construct a judgment matrix based on the ratio between each pair of the normalized discrete deviation degrees.

[0123] S620: Based on the judgment matrix, calculate the weights corresponding to the feature subsets and participants, and correct the feature vectors according to the weights.

[0124] For example: After deleting outliers from the discrete deviation set corresponding to a feature subset, the corresponding discrete deviation set λ m ={λ 1m ,λ 2m ,...λ nm}, for set λ mNormalization is performed to ensure that all discrete deviations are mapped to the same coordinate system, while ensuring that feature vectors with higher discrete deviations have lower weights in the calculation, and feature vectors with lower discrete deviations have higher weights. Optionally, the normalization formula is:

[0125]

[0126] Therefore, according to S610, the judgment matrix υ is:

[0127]

[0128] In the training process of a horizontal federated learning model, all participants use the same training algorithm, differing only in the raw data used for training. Therefore, it can be assumed that the feature vectors calculated by each participant follow a linear trend. Optionally, the weights ω of each feature vector... i The calculation formula is as follows:

[0129]

[0130] Then the m-th eigenvector a km After correction, it becomes w km *a km The sum of the data values ​​for a subset of features with m eigenvectors is:

[0131]

[0132] The average corrected feature vector of each feature subset can be calculated by summing the data values, and then the federated learning model can be trained based on this average corrected feature vector. Those skilled in the art will understand that during the federated learning model training process, the operation of receiving feature sets sent by participants, summarizing the feature vectors in the feature sets to obtain corrected feature vectors and / or average corrected feature vectors, and feeding back the corrected feature vectors and / or average corrected feature vectors to each participant may need to be performed multiple times. Therefore, the data processing method provided in this application may be executed multiple times during the federated learning model training process until the federated learning model training is completed.

[0133] Based on the data processing method of the federated learning model provided in the above embodiments, this application also provides specific implementations of the data processing apparatus. Please refer to the following embodiments.

[0134] First see Figure 9 The data processing apparatus 900 provided in this application embodiment includes the following modules:

[0135] The acquisition module 901 is used to acquire the feature set sent by the participants of the federated learning model. The feature set includes at least one feature subset, and the feature subset includes multiple feature vectors.

[0136] Calculation module 902 is used to calculate the standard deviation of the samples corresponding to each feature subset based on the feature vector;

[0137] The calculation module 902 is also used to calculate the discrete deviation set corresponding to each feature subset based on the feature subset and the standard deviation of the sample corresponding to it. The discrete deviation set includes the discrete deviation that corresponds one-to-one with each feature vector.

[0138] The identification module 903 is used to perform clustering processing on the discrete deviation set, determine the abnormal deviation in the discrete deviation set based on the clustering processing results, and identify the feature vector corresponding to the abnormal deviation.

[0139] The device can calculate the sample standard deviation for each feature subset, and then calculate the discrete deviation corresponding to the feature vectors of each feature subset. Thus, the discrete deviation reflects the degree of dispersion of each feature vector. Furthermore, the device can perform clustering on the discrete deviation set, and determine the abnormal deviation in the discrete deviation set based on the clustering results. This allows the device to identify the feature vectors corresponding to the abnormal deviations, and further perform processing such as removing, correcting, and reducing the weights of the feature vectors corresponding to the abnormal deviations to improve the accuracy of the feature vector aggregation calculation.

[0140] As one implementation of this application, the aforementioned identification module 903 includes:

[0141] Clustering units are used to determine the coverage area with each discrete offset as the cluster center for each set of discrete offsets corresponding to a feature subset.

[0142] The aggregation factor calculation unit is used to calculate the aggregation factor corresponding to each coverage area based on the similarity between each discrete offset and the cluster center.

[0143] The anomaly identification unit is used to determine the abnormal deviation degree in the discrete deviation degree set according to the aggregation factor corresponding to each coverage area, and to identify the feature vector corresponding to the abnormal deviation degree.

[0144] As one implementation of this application, the above clustering unit is used for:

[0145] For each discrete offset set corresponding to a feature subset, each discrete offset is set as a cluster center, and the Euclidean distance between the cluster center and other discrete offsets is sorted.

[0146] Set the Euclidean distance in the preset order to the domain radius corresponding to the cluster center;

[0147] Set the set of discrete offsets whose Euclidean distance is less than the domain radius as the coverage area corresponding to the cluster center.

[0148] As one implementation of this application, the above clustering unit is used for:

[0149] For each coverage area, the average discrete distance to the coverage area is calculated based on the similarity between each discrete offset in the coverage area and the cluster center.

[0150] Based on the average discrete distance, the discrete density corresponding to each cluster center is calculated, and the discrete density corresponds one-to-one with the discrete offset.

[0151] For each coverage area, calculate the density ratio of other discrete offsets in the coverage area to which the cluster center belongs to the cluster center, and calculate the average density ratio based on the density ratio. Set the average density ratio as the aggregation factor.

[0152] As one implementation of this application, the above-mentioned aggregation factor calculation unit is used for:

[0153] For each discrete offset in the coverage area, determine the maximum value between the discrete distance between the cluster center and the discrete offset, and the Euclidean distance between the cluster center and the discrete offset, and set the maximum value as the similarity between the cluster center and the discrete offset.

[0154] The average discrete distance corresponding to the coverage area is calculated based on the similarity between the cluster centers and discrete offsets in the coverage area.

[0155] As one implementation of this application, the above-mentioned computing module 902 includes:

[0156] The feature mean unit is used to calculate the feature mean corresponding to each feature subset based on the feature vector.

[0157] The standard deviation unit is used to calculate the sample standard deviation of each feature subset based on the feature vector and feature mean corresponding to each feature subset.

[0158] As one implementation of this application, the above-mentioned computing module 902 includes:

[0159] The discrete offset unit is used to calculate the discrete offset corresponding to each feature vector based on the feature vector, feature mean, and sample standard deviation of each feature subset.

[0160] Among them, the discrete offsets corresponding to the feature vectors in the same feature subset form a discrete deviation set.

[0161] As one implementation of this application, the above-mentioned device also includes:

[0162] The cleaning module 904 is used to delete the feature vectors corresponding to the abnormal deviations in the feature set and to delete the abnormal deviations in the discrete deviation set.

[0163] The correction module 905 is used to calculate the weights corresponding to each feature vector based on the discrete deviation set, and to correct the feature vectors based on the weights.

[0164] As one implementation of this application, the above-mentioned correction module 905 includes:

[0165] The judgment matrix unit is used to normalize the discrete deviation degree corresponding to a feature subset, and construct the judgment matrix based on the ratio between each pair of the normalized discrete deviation degrees.

[0166] The correction unit is used to calculate the weights corresponding to the feature subsets and participants based on the judgment matrix, and to correct the feature vectors based on the weights.

[0167] The data processing apparatus provided in this embodiment of the invention can achieve Figures 1 to 8 To avoid repetition, the steps in the method embodiments will not be described again here.

[0168] Figure 10 A schematic diagram of the hardware structure of the data processing device provided in an embodiment of this application is shown.

[0169] The data processing device may include a processor 1001 and a memory 1002 storing computer program instructions.

[0170] Specifically, the processor 1001 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0171] Memory 1002 may include mass storage for data or instructions. For example, and not limitingly, memory 1002 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 1002 may include removable or non-removable (or fixed) media. Where appropriate, memory 1002 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 1002 is non-volatile solid-state memory.

[0172] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to one aspect of this disclosure.

[0173] The processor 1001 reads and executes computer program instructions stored in the memory 1002 to implement the data processing method of any of the federated learning models in the above embodiments.

[0174] In one example, the data processing device may further include a communication interface 1003 and a bus 1010. Wherein, for example... Figure 10 As shown, the processor 1001, memory 1002, and communication interface 1003 are connected through bus 1010 and complete communication with each other.

[0175] The communication interface 1003 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0176] Bus 1010 includes hardware, software, or both, that couples components of a data processing device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 1010 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0177] The data processing device can be based on the above embodiments to achieve the combination Figures 1 to 9 The data processing methods and apparatus described in the federated learning model.

[0178] Furthermore, in conjunction with the data processing method of the federated learning model in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the data processing methods of the federated learning model in the above embodiments and achieve the same technical effect. To avoid repetition, further details are omitted here. The aforementioned computer-readable storage medium may include non-transitory computer-readable storage media, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, etc., and is not limited thereto.

[0179] In addition, this application also provides a computer program product, including computer program instructions, which, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0180] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0181] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0182] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0183] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0184] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A data processing method for a federated learning model, characterized in that, The method includes: Obtain a feature set sent by the participants in the federated learning model, wherein the participants are communication operators; the feature set includes at least one feature subset, and the feature subset includes multiple feature vectors; the feature vectors are generated by the participants based on physical data collected by local data acquisition devices; the feature vectors include a communication active time vector and a communication active address vector. For each of the feature subsets, the standard deviation of the samples corresponding to each feature subset is calculated based on the feature vector; Based on the feature subset and the standard deviation of the sample corresponding to it, calculate the discrete deviation set corresponding to each feature subset, the discrete deviation set including the discrete deviation corresponding to each feature vector; The discrete deviation set is clustered, and the abnormal deviation in the discrete deviation set is determined based on the clustering results, and the feature vector corresponding to the abnormal deviation is identified. The feature vector corresponding to the abnormal deviation is deleted from the feature set, and the abnormal deviation is deleted from the discrete deviation set. The weights corresponding to each feature vector are calculated based on the set of discrete deviations, and the feature vectors are corrected based on the weights.

2. The data processing method for the federated learning model according to claim 1, characterized in that, The process of clustering the discrete deviation set, determining the abnormal deviations in the discrete deviation set based on the clustering results, and identifying the feature vectors corresponding to the abnormal deviations includes: For each discrete deviation set corresponding to the feature subset, a coverage area with each discrete deviation as the cluster center is determined. For each coverage area, a aggregation factor corresponding to the coverage area is calculated based on the similarity between each discrete offset and the cluster center; Based on the aggregation factor corresponding to each of the coverage areas, the abnormal deviation degree in the discrete deviation degree set is determined, and the feature vector corresponding to the abnormal deviation degree is identified.

3. The data processing method for the federated learning model according to claim 2, characterized in that, Each discrete deviation set corresponding to the feature subset determines a coverage area with each discrete deviation as the cluster center, including: For each discrete deviation set corresponding to the feature subset, each discrete deviation is set as a cluster center, and the Euclidean distance between the cluster center and other discrete deviations is sorted. Set the Euclidean distance in the preset order as the domain radius corresponding to the cluster center; Set the set of discrete offsets whose Euclidean distance is less than the domain radius as the coverage domain corresponding to the cluster center.

4. The data processing method for the federated learning model according to claim 2, characterized in that, For each coverage region, the aggregation factor corresponding to the coverage region is calculated based on the similarity between each discrete offset and the cluster centers, including: For each coverage area, the average discrete distance corresponding to the coverage area is calculated based on the similarity between each discrete offset in the coverage area and the cluster center. Based on the average discrete distance, the discrete density corresponding to each of the cluster centers is calculated, and the discrete density corresponds one-to-one with the discrete offset. For each coverage area, the density ratio of other discrete offsets in the coverage area to which the cluster center belongs is calculated to that of the cluster center, and the average density ratio is calculated based on the density ratio. The average density ratio is then set as the aggregation factor corresponding to the cluster center.

5. The data processing method for the federated learning model according to claim 4, characterized in that, For each coverage area, the average discrete distance corresponding to the coverage area is calculated based on the similarity between each discrete offset in the coverage area and the cluster center, including: For each discrete offset in the coverage area, determine the maximum value of the discrete distance between the cluster center and the discrete offset, and the Euclidean distance between the cluster center and the discrete offset, and set the maximum value as the similarity between the cluster center and the discrete offset; The average discrete distance corresponding to the coverage area is calculated based on the similarity between the cluster centers and discrete offsets in the coverage area.

6. The data processing method for the federated learning model according to claim 1, characterized in that, The step of calculating the sample standard deviation corresponding to each feature subset based on the feature vector includes: For each feature subset, the mean value of the feature corresponding to the feature subset is calculated based on the feature vector; The sample standard deviation of each feature subset is calculated based on the feature vector and the feature mean corresponding to each feature subset.

7. The data processing method for the federated learning model according to claim 6, characterized in that, The step of calculating the set of discrete deviations based on the feature subset and the corresponding sample standard deviation includes: Based on the feature vector, the feature mean, and the sample standard deviation corresponding to each feature subset, calculate the discrete offset corresponding to each feature vector. The discrete offsets corresponding to the feature vectors in the same feature subset form the discrete offset set.

8. The data processing method for the federated learning model according to claim 1, characterized in that, The step of calculating the weights corresponding to each feature vector based on the set of discrete deviations, and correcting the feature vectors based on the weights, includes: For the discrete deviation degree corresponding to the aforementioned feature subset, normalization processing is performed, and a judgment matrix is ​​constructed based on the ratio between each pair of the normalized discrete deviation degrees. Based on the judgment matrix, calculate the weights corresponding to the feature subset and the participants, and correct the feature vectors according to the weights.

9. A data processing apparatus, characterized in that, The data processing device includes: The acquisition module is used to acquire a feature set sent by the participants of the federated learning model, wherein the participants are communication operators; the feature set includes at least one feature subset, and the feature subset includes multiple feature vectors; the feature vectors are generated by the participants based on physical data collected by local data acquisition devices; the feature vectors include a communication active time vector and a communication active address vector. The calculation module is used to calculate the sample standard deviation corresponding to each feature subset based on the feature vector; The calculation module is also used to calculate the discrete deviation set corresponding to each of the feature subsets based on the feature subsets and the standard deviation of the samples corresponding to them, wherein the discrete deviation set includes discrete deviations that correspond one-to-one with each of the feature vectors; The identification module is used to perform clustering processing on the discrete deviation set, determine the abnormal deviation in the discrete deviation set according to the clustering processing result, and identify the feature vector corresponding to the abnormal deviation. The deletion module is used to delete the feature vector corresponding to the abnormal deviation from the feature set, and to delete the abnormal deviation from the discrete deviation set. The correction module is used to calculate the weights corresponding to each of the feature vectors based on the discrete deviation set, and to correct the feature vectors based on the weights.

10. A data processing device, characterized in that, The data processing device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the data processing method of the federated learning model as described in any one of claims 1-9.

11. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, implement the data processing method of the federated learning model as described in any one of claims 1-9.

12. A computer program product, characterized in that, The computer program product includes computer program instructions that, when executed by a processor, implement the data processing method of the federated learning model as described in any one of claims 1-9.

Images