Methods and apparatus for detecting and correcting federated learning data

Abstract

This invention relates to a method and apparatus for detecting and correcting federated learning data. The method includes: acquiring intermediate data uploaded by each participating party, the intermediate data including location information of data to be supplemented, suspicious data, and indicators related to the suspicious data; performing a global analysis based on the indicators related to the suspicious data to filter the suspicious data and determine the data to be corrected; determining a problem data matrix based on the location information of the data to be supplemented and the data to be corrected; acquiring the associated data corresponding to each problem data in the problem data matrix from each participating party, analyzing the associated data, and determining a correction matrix for each participating party, so as to provide data processing services based on the correction matrix of the participating parties and the local data of the participating parties. The technical solution provided by this invention can effectively solve the technical problems of data accuracy, consistency, and security by combining data quality, data validity detection, and data protection mechanisms.

Description

Technical Field

[0001] This invention relates to the field of privacy data processing technology, and in particular to a method and apparatus for detecting and correcting federated learning data. Background Technology

[0002] For machine learning, high-quality data often leads to models with better generalization performance. Federated learning, as a decentralized distributed machine learning method, achieves data availability without visibility while using data distributed across different machines or data centers to build machine learning models. Since the data comes from various participants, it is also necessary to ensure that the original data is not known to other participants while implementing the model. However, the local data of each participant is often imperfect (e.g., containing outliers, missing values, etc.). If this data is directly used for model training, poor modeling results may be achieved. Especially when there is a large amount of problematic data, it will directly affect the overall modeling effect, and in severe cases, cause modeling bias. This not only leads to incorrect decisions but also wastes computing space. Summary of the Invention

[0003] Based on the above-mentioned situation of the prior art, the purpose of this embodiment of the invention is to provide a method and apparatus for detecting and correcting federated learning data. While ensuring data security and privacy, the server uses the intermediate results of each participating end to evaluate the availability and validity of the original data, and then uses the federated learning method to analyze the data, thereby achieving data correction.

[0004] To achieve the above objectives, according to a first aspect of the present invention, a method for detecting and correcting federated learning data is provided, applied on a server side, comprising:

[0005] Obtain intermediate data uploaded by each participating party, including location information of data to be supplemented, suspicious data, and indicators related to the suspicious data;

[0006] A global analysis is performed based on the indicators related to the suspicious data to filter out the suspicious data and determine the data to be corrected;

[0007] Based on the location information of the data to be supplemented and the data to be corrected, determine the problem data matrix;

[0008] The system obtains the associated data corresponding to each problem data in the problem data matrix from each participating party, analyzes the associated data, determines the correction matrix for each participating party, and provides data processing services based on the correction matrix and local data of each participating party.

[0009] Furthermore, based on the indicators related to the suspicious data, a global analysis is performed to filter the suspicious data and determine the data to be corrected, including:

[0010] Identify global metrics related to suspicious data from each participating party.

[0011] Based on the global indicators, determine the function that the suspicious data matches, and determine the degree of anomaly of the suspicious data based on the matching between the suspicious data and the function;

[0012] Based on the degree of abnormality of the suspicious data, select the data to be corrected from the suspicious data.

[0013] Furthermore, global metrics related to suspicious data from each participating party are identified, including:

[0014] Based on the data from each participating party and the indicators corresponding to the suspicious data, global indicators are determined; the indicators include at least one of the data's mean, maximum value, minimum value, variance, and mode.

[0015] Furthermore, the step of obtaining the associated data corresponding to each problem data in the problem data matrix from each participating party, and analyzing the associated data to determine the correction matrix for each participating party includes:

[0016] For data to be supplemented, obtain the filling data from the associated data and add it to the correction matrix;

[0017] For the data to be corrected, relevant data is obtained from the associated data and input into the analysis model to determine the correction value, and the correction value is added to the correction matrix. The analysis model is trained using federated learning.

[0018] Furthermore, the method also includes analyzing the training process of the model:

[0019] The model gradients of each participant are obtained. The model gradients are obtained by training the local training data of each participant. The local training data is obtained by random data missing and random data adjustment. The labeled data of the local training data is the data before random missing and random adjustment.

[0020] Based on the model gradient, update the model parameters of the analysis model until the model parameters converge.

[0021] Furthermore, after determining the problem data matrix, the method also includes:

[0022] The homomorphically encrypted data matrix to be supplemented is sent to each participating party to determine the missing values ​​that can be filled based on the feedback from each participating party, and these values ​​are marked so that data repair can be performed based on the markings.

[0023] The homomorphically encrypted data matrix to be corrected is sent to each participating party to determine the correctable data based on the feedback from each participating party and to mark it so that the data can be repaired based on the marking.

[0024] The data matrix to be supplemented and the data matrix to be corrected are obtained based on the correction matrix.

[0025] Furthermore, the method also includes:

[0026] The trusted execution environment of the participating party is determined, and the correction matrix is ​​sent to the participating party so that the participating party can fuse local data with the correction matrix in its local trusted execution environment and perform data processing based on the fused data.

[0027] Furthermore, the method also includes:

[0028] Receive the initial analysis results and data to be analyzed from the participating parties;

[0029] In a trusted execution environment, the second analysis result is determined based on the correction matrix and the data to be analyzed;

[0030] In a trusted execution environment, the data processing results are determined based on the results of the first and second analyses.

[0031] Furthermore, the method also includes:

[0032] Send a homomorphic encryption algorithm to the participating party so that the participating party can encrypt its local data according to the homomorphic encryption algorithm;

[0033] The homomorphic encryption algorithm is used to encrypt the repaired data, and the encrypted data is sent to the participating party so that the participating party can use its local model to train the data and obtain encrypted training results and encrypted labels.

[0034] The system receives encrypted training results and encrypted labels from the participating party, calculates the difference between the encrypted training results and encrypted labels, and then decrypts them using a homomorphic encryption algorithm to adjust the model on the participating party's end.

[0035] According to a second aspect of the present invention, a method for detecting and correcting federated learning data is provided, applied to the participating party, comprising:

[0036] The participating party performs calculations on local data, analyzes the data to be supplemented, and determines the location information of the data to be supplemented.

[0037] The participating party calculates indicators for local data locally and identifies suspicious data based on these indicators.

[0038] Intermediate data is determined based on the location information of the data to be supplemented, the suspicious data, and the indicators related to the suspicious data;

[0039] Upload the intermediate data to the server.

[0040] According to a third aspect of the present invention, a detection and correction apparatus for federated learning data is provided, applied on a server side, comprising:

[0041] The intermediate data acquisition module is used to acquire intermediate data uploaded by each participating party. The intermediate data includes the location information of the data to be supplemented, suspicious data, and indicators related to the suspicious data.

[0042] The data to be corrected determination module is used to perform a global analysis based on the indicators related to the suspicious data, filter the suspicious data, and determine the data to be corrected.

[0043] The problem data matrix determination module is used to determine the problem data matrix based on the location information of the data to be supplemented and the data to be corrected.

[0044] The correction matrix determination module is used to obtain the associated data corresponding to each problem data in the problem data matrix from each participant's end, analyze the associated data, determine the correction matrix for each participant's end, and provide data processing services based on the correction matrix of the participant's end and the local data of the participant's end.

[0045] In summary, this invention provides a method and apparatus for detecting and correcting federated learning data. The method includes: acquiring intermediate data uploaded by each participating party, the intermediate data including location information of data to be supplemented, suspicious data, and indicators related to the suspicious data; performing global analysis based on the indicators related to the suspicious data, filtering the suspicious data, and determining the data to be corrected; determining a problem data matrix based on the location information of the data to be supplemented and the data to be corrected; acquiring the associated data corresponding to each problem data in the problem data matrix from each participating party, analyzing the associated data, and determining the correction matrix for each participating party, so as to provide data processing services based on the correction matrix of the participating party and the local data of the participating party. The technical solution provided by this invention, based on the intermediate data calculated by the participating party using rules locally, encrypts it and analyzes it using a federated learning method. Then, the global feature distribution information calculated by each participating party using federated learning is stored in a trusted execution environment and corrected according to the selected problem data. It can effectively solve the technical problems of data accuracy, consistency, and security by combining data quality, data validity detection, and data protection mechanisms. It is suitable for organizations with strict privacy restrictions, such as data processing in fields such as healthcare and finance. Attached Figure Description

[0046] Figure 1This is a flowchart of a method for detecting and correcting federated learning data according to an embodiment of the present invention;

[0047] Figure 2 This is a flowchart of a method for detecting and correcting federated learning data provided in another embodiment of the present invention;

[0048] Figure 3 This is an overall flowchart of the federated learning data detection and correction method provided in this embodiment of the invention;

[0049] Figure 4 This is a block diagram of a federated learning data detection and correction device provided in one embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0052] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of the present invention should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in one or more embodiments of the present invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the element or object listed following the word and its equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0053] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings. An embodiment of the present invention provides a method for detecting and correcting federated learning data, applied on the server side. Figure 1 The flowchart of the method is shown below, which includes the following steps:

[0054] S202. Obtain intermediate data uploaded by each participating party. This intermediate data includes location information of data to be supplemented, suspicious data, and indicators related to the suspicious data. The intermediate data uploaded by the participating parties mainly includes two types: missing data that needs to be supplemented, and data that needs to be corrected. Data to be supplemented can be determined by each participating party through local searching and calculation. Each participating party calculates and analyzes its local data to identify the data to be supplemented and determines its location information, then uploads this location information as intermediate data. Data requiring correction cannot be determined by the participating parties and therefore needs to be determined on the server side. Each participating party calculates indicators for its local data and determines suspicious data based on these indicators. These indicators include, but are not limited to, the mean, maximum, minimum, variance, and mode of the data. The suspicious data and its related indicators are then uploaded as intermediate data.

[0055] S204. Perform a global analysis based on the indicators related to the suspicious data, filter the suspicious data, and determine the data to be corrected. This step can be performed as follows: determine the global indicators related to the suspicious data from each participant's end; determine the global indicators by matching the data from each participant's end with the indicators corresponding to the suspicious data; determine the function that the suspicious data conforms to based on the global indicators; determine the anomaly degree of the suspicious data based on the matching between the suspicious data and the function; and filter the data to be corrected from the suspicious data based on the anomaly degree of the suspicious data. The function that the suspicious data conforms to based on the global indicators can be a Gaussian distribution, an exponential distribution, a multivariate Gaussian distribution, etc. Taking a Gaussian distribution as an example, based on various indicators of the data, such as maximum value, mean, and variance, determine whether the data conforms to a Gaussian distribution, thereby selecting data that does not conform to a Gaussian distribution. If it is data from a single participant's end, a certain data point may not conform to a Gaussian distribution, but this data point conforms to the global Gaussian distribution, thus allowing for more accurate filtering of anomalous data.

[0056] S206. Based on the location information of the data to be supplemented and the data to be corrected, determine the problem data matrix. This problem data matrix indicates the location information of the data to be supplemented and the data that needs to be corrected. The server can obtain relevant data from each participating party based on this problem matrix.

[0057] S208. Obtain the associated data corresponding to each problem data in the problem data matrix from each participating party, analyze the associated data, and determine the correction matrix for each participating party. Provide data processing services based on the correction matrix and local data of each participating party. Determining the correction matrix for each participating party includes: for data to be supplemented, obtaining filler data from the associated data and adding it to the correction matrix; for data to be corrected, obtaining relevant data from the associated data and inputting it into the analysis model, determining the correction value, and adding the correction value to the correction matrix. The analysis model is trained using federated learning. Based on the associated data corresponding to each problem data in the problem data matrix obtained from each participating party, a correction method is then determined. The associated data can be obtained through unintentional transmission. For example, data obtained from each participating party may include the mean of the feature (an example scheme involves fluctuations around the mean), and other indicators. The data obtained from each participating party may also include information about other features related to that feature from that participating party. This part of the content can be obtained through unintentional transmission, ensuring that the participating party is unaware of the specific content of the obtained data. The server then completes data filling and correction based on the obtained data in a trusted execution environment.

[0058] According to certain optional embodiments, the analysis model is trained through the following steps: obtaining the model gradients of each participant, which are obtained through training on local training data of each participant, wherein the local training data is obtained through random data missing and random data adjustment, and the labeled data of the local training data is the data before random missing and random adjustment; updating the model parameters of the analysis model based on the model gradients until the model parameters converge. This model training process relies on federated learning for iterative training.

[0059] According to certain optional embodiments, after determining the problem data matrix, the method further includes:

[0060] The homomorphically encrypted data matrix to be supplemented is sent to each participating party to determine the missing values ​​that can be filled based on the feedback from each participating party, and these values ​​are marked so that data repair can be performed based on the markings.

[0061] The homomorphically encrypted data matrix to be corrected is sent to each participating party to determine the correctable data based on the feedback from each participating party and to mark it so that the data can be repaired based on the marking.

[0062] The data matrix to be supplemented and the data matrix to be corrected are obtained based on the correction matrix.

[0063] According to some optional embodiments, the method further includes: determining a trusted execution environment for a participating party, sending a correction matrix to the participating party, so that the participating party can fuse local data with the correction matrix in its local trusted execution environment, and perform data processing based on the fused data. This can be used for example, model training, big data analysis, and user evaluation. The corrected data is only computed in plaintext within the trusted execution environment; in other cases, it can be deleted or stored in encrypted form.

[0064] According to some optional embodiments, the method further includes: receiving a first analysis result and data to be analyzed from the participant; determining a second analysis result based on a correction matrix and the data to be analyzed in a trusted execution environment; and determining a data processing result based on the first analysis result and the second analysis result in the trusted execution environment.

[0065] According to some optional embodiments, the method further includes: sending a homomorphic encryption algorithm to the participating party, so that the participating party encrypts its local data according to the homomorphic encryption algorithm; encrypting the repaired data using the homomorphic encryption algorithm, and sending the encrypted data to the participating party, so that the participating party trains the data using its local model to obtain encrypted training results and encrypted labels; receiving the encrypted training results and encrypted labels sent by the participating party, calculating the difference between the encrypted training results and encrypted labels, and decrypting them using the homomorphic encryption algorithm to adjust the model on the participating party. Since the labels are also provided to the participating party using homomorphic encryption, neither the labels nor the repaired content are leaked.

[0066] According to certain optional embodiments, the sample set can also be expanded using a global feature distribution model based on the data feature expansion requirements of the participating parties, and then returned to each participating party.

[0067] According to certain optional embodiments, before performing data validity detection and correction, a contract can be customized with each participating party, declaring the dataset information used for detection and other data processing requirements of each participating party. The server can only initiate the task instruction after all participating parties have acknowledged and agreed to the terms. If a participating party has doubts about the purpose of this task, they can modify the contract and re-initiate the contract request.

[0068] Embodiments of the present invention also provide a method for detecting and correcting federated learning data, applied to the participating party. Figure 2 The flowchart of the method is shown below, which includes the following steps:

[0069] S402. Calculate the local data on the participating party's end, analyze the data to be supplemented, and determine the location information of the data to be supplemented.

[0070] S404. Calculate the indicators of local data on the participating party's end, and determine suspicious data based on the indicators.

[0071] S406. Based on the location information of the data to be supplemented, the suspicious data, and the indicators related to the suspicious data, determine the intermediate data.

[0072] S408. Upload the intermediate data to the server.

[0073] Figure 3 The diagram illustrates the overall flowchart of the federated learning data detection and correction method according to an embodiment of the present invention, integrating the processing flow of the server side and each participating party. Figure 3 As shown, the server selects data participants for detection and correction based on requirements, and initiates data validity detection and correction requests and task contracts to each participant to confirm whether all participants have confirmed the contract content. Upon confirmation, data integrity and validity detection begins. If any participant has doubts about the contract content, the contract can be modified and the request resubmitted. The detection and correction process is as described in the above embodiment of the invention: each participant analyzes locally to obtain indicators of suspicious data and location information of data to be supplemented, and uploads them to the server. The server performs global feature calculations to determine the problem matrix and obtains corresponding data from each participant for supplementation and correction. Figure 3 The example given uses two participating parties. The technical solution of this invention also includes federated computing with more participating parties.

[0074] Embodiments of the present invention also provide a detection and correction device for federated learning data, applied on the server side. Figure 4 The diagram shows the configuration of the device, including:

[0075] The intermediate data acquisition module 401 is used to acquire intermediate data uploaded by each participating party. The intermediate data includes the location information of the data to be supplemented, suspicious data, and indicators related to the suspicious data.

[0076] The data to be corrected determination module 402 is used to perform a global analysis based on the indicators related to the suspicious data, filter the suspicious data, and determine the data to be corrected.

[0077] Problem data matrix determination module 403 is used to determine the problem data matrix based on the location information of the data to be supplemented and the data to be corrected;

[0078] The correction matrix determination module 404 is used to obtain the associated data corresponding to each problem data in the problem data matrix from each participating party, analyze the associated data, determine the correction matrix of each participating party, and provide data processing services based on the correction matrix of the participating party and the local data of the participating party.

[0079] The specific process by which each module in the federated learning data detection and correction device provided in the above embodiments of the present invention implements its function is the same as the steps of the federated learning data detection and correction method provided in the above embodiments of the present invention. Therefore, the repeated description will be omitted here.

[0080] An embodiment of the present invention also provides an electronic device. Figure 5 The diagram shown is a structural schematic of an electronic device provided according to an embodiment of the present invention. Figure 5 As shown, the electronic device 500 includes: one or more processors 501 and a memory 502; and computer program instructions stored in the memory 502, which, when executed by the processor 501, cause the processor 501 to perform the detection and correction method for federated learning data as described in any of the above embodiments. The processor 501 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0081] The memory 502 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1001 may execute the program instructions to implement the steps in the federated learning data detection and correction methods of the various embodiments of the present invention described above, and / or other desired functions.

[0082] In some embodiments, the electronic device 500 may further include an input device 503 and an output device 504, these components being connected via a bus system and / or other forms of connection mechanisms. Figure 5 (Not shown in the diagram) interconnected. For example, when the electronic device is a standalone device, the input device 503 can be a communication network connector for receiving acquired input signals from external mobile devices. Furthermore, the input device 503 may also include, for example, a keyboard, mouse, microphone, etc. The output device 504 can output various information to the outside, and may include, for example, a monitor, speaker, printer, and communication network and its connected remote output devices.

[0083] In addition to the methods and devices described above, embodiments of the present invention may also be computer program products, including computer program instructions, which, when executed by a processor, cause the processor to perform the steps in the federated learning data detection and correction method as described in any of the above embodiments.

[0084] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of the present invention. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0085] Furthermore, embodiments of the present invention may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the federated learning data detection and correction method of various embodiments of the present invention.

[0086] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0087] It should be understood that the processor in the embodiments of the present invention can be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0088] In summary, this invention relates to a method and apparatus for detecting and correcting federated learning data. The method includes: acquiring intermediate data uploaded by each participating party, the intermediate data including location information of data to be supplemented, suspicious data, and indicators related to the suspicious data; performing global analysis based on the indicators related to the suspicious data, filtering the suspicious data, and determining the data to be corrected; determining a problem data matrix based on the location information of the data to be supplemented and the data to be corrected; acquiring the associated data corresponding to each problem data in the problem data matrix from each participating party, analyzing the associated data, and determining the correction matrix for each participating party, so as to provide data processing services based on the correction matrix of the participating party and the local data of the participating party. The technical solution provided by this invention, based on the intermediate data calculated by the participating party using rules locally, encrypts it and analyzes it using a federated learning method. Then, the global feature distribution information calculated by each participating party using federated learning is stored in a trusted execution environment and corrected according to the selected problem data. It can effectively solve the technical problems of data accuracy, consistency, and security by combining data quality, data validity detection, and data protection mechanisms. It is suitable for organizations with strict privacy restrictions, such as data processing in fields such as healthcare and finance. Furthermore, because federated learning only uses a portion of the intermediate information in the data, it can reduce the latency of computing large amounts of data. The calculated global data distribution information can also provide more high-quality data for participants with sparse samples, further enriching the dataset sample.

[0089] It should be understood that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of this invention, technical features of the above embodiments or different embodiments can also be combined, steps can be implemented in any order, and many other variations exist regarding different aspects of one or more embodiments of the invention as described above; for the sake of brevity, they are not provided in the details. The specific embodiments described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A method for detecting and correcting federated learning data, characterized in that, Applied to the server side, including: Obtain intermediate data uploaded by each participating party, including location information of data to be supplemented, suspicious data, and indicators related to the suspicious data; A global analysis is performed based on the indicators related to the suspicious data to filter out the suspicious data and determine the data to be corrected; Based on the location information of the data to be supplemented and the data to be corrected, determine the problem data matrix; The process involves obtaining the associated data corresponding to each problem data in the problem data matrix from each participating party, analyzing the associated data, and determining the correction matrix for each participating party. This includes: for data to be supplemented, obtaining the filling data from the associated data and adding it to the correction matrix; for data to be corrected, obtaining the relevant data from the associated data, inputting it into the analysis model, determining the correction value, and adding the correction value to the correction matrix. Provide data processing services based on the correction matrix of the participant and the local data of the participant, including: determining the trusted execution environment of the participant, sending the correction matrix to the participant so that the participant can fuse the local data with the correction matrix in the local trusted execution environment, and performing data processing based on the fused data.

2. The method according to claim 1, characterized in that, A global analysis is performed based on the indicators related to the suspicious data to filter the suspicious data and determine the data to be corrected, including: Identify global metrics related to suspicious data from each participating party. Based on the global indicators, determine the function that the suspicious data matches, and determine the degree of anomaly of the suspicious data based on the matching between the suspicious data and the function; Based on the degree of abnormality of the suspicious data, select the data to be corrected from the suspicious data.

3. The method according to claim 2, characterized in that, Global metrics for identifying suspicious data from each participating party, including: Based on the data from each participating party and the indicators corresponding to the suspicious data, global indicators are determined; the global indicators include at least one of the data's mean, maximum value, minimum value, variance, and mode.

4. The method according to claim 3, characterized in that, The analytical model was trained using a federated learning approach.

5. The method according to claim 4, characterized in that, The method also includes analyzing the training process of the model: The model gradients of each participant are obtained. The model gradients are obtained by training the local training data of each participant. The local training data is obtained by random data missing and random data adjustment. The labeled data of the local training data is the data before random missing and random adjustment. Based on the model gradient, update the model parameters of the analysis model until the model parameters converge.

6. The method according to claim 5, characterized in that, After determining the problem data matrix, the method further includes: The homomorphically encrypted data matrix to be supplemented is sent to each participating party to determine the missing values ​​that can be filled based on the feedback from each participating party, and these values ​​are marked so that data repair can be performed based on the markings. The homomorphically encrypted data matrix to be corrected is sent to each participating party to determine the correctable data based on the feedback from each participating party and to mark it so that the data can be repaired based on the marking. The data matrix to be supplemented and the data matrix to be corrected are obtained based on the correction matrix.

7. The method according to claim 6, characterized in that, The method further includes: Receive the initial analysis results and data to be analyzed from the participating parties; In a trusted execution environment, the second analysis result is determined based on the correction matrix and the data to be analyzed; In a trusted execution environment, the data processing results are determined based on the results of the first and second analyses.

8. The method according to claim 7, characterized in that, The method further includes: Send a homomorphic encryption algorithm to the participating party so that the participating party can encrypt its local data according to the homomorphic encryption algorithm; The homomorphic encryption algorithm is used to encrypt the repaired data, and the encrypted data is sent to the participating party so that the participating party can use its local model to train the data and obtain encrypted training results and encrypted labels. The system receives encrypted training results and encrypted labels from the participating party, calculates the difference between the encrypted training results and encrypted labels, and then decrypts them using a homomorphic encryption algorithm to adjust the model on the participating party's end.

9. A method for detecting and correcting federated learning data, characterized in that, Applied to the participating parties, including: The participating party performs calculations on local data, analyzes the data to be supplemented, and determines the location information of the data to be supplemented. The participating party calculates indicators for local data locally and identifies suspicious data based on these indicators. Intermediate data is determined based on the location information of the data to be supplemented, the suspicious data, and the indicators related to the suspicious data; The intermediate data is uploaded to the server so that the server can perform a global analysis based on the indicators related to the suspicious data, filter the suspicious data, determine the data to be corrected, and determine the problem data matrix based on the location information of the data to be supplemented and the data to be corrected. The system uploads the associated data corresponding to each problem data in the problem data matrix to the server so that the server can analyze the associated data and determine the correction matrix for each participating party. This includes: for data to be supplemented, obtaining the filling data from the associated data and adding it to the correction matrix; for data to be corrected, obtaining the relevant data from the associated data and inputting it into the analysis model to determine the correction value and adding the correction value to the correction matrix. It receives the correction matrix sent by the server, merges the local data with the correction matrix in the local trusted execution environment, and performs data processing based on the merged data.

10. A device for detecting and correcting federated learning data, characterized in that, Applied to the server side, including: The intermediate data acquisition module is used to acquire intermediate data uploaded by each participating party. The intermediate data includes the location information of the data to be supplemented, suspicious data, and indicators related to the suspicious data. The data to be corrected determination module is used to perform a global analysis based on the indicators related to the suspicious data, filter the suspicious data, and determine the data to be corrected. The problem data matrix determination module is used to determine the problem data matrix based on the location information of the data to be supplemented and the data to be corrected. The correction matrix determination module is used to obtain the associated data corresponding to each problem data in the problem data matrix from each participating party, analyze the associated data, and determine the correction matrix for each participating party. This includes: for data to be supplemented, obtaining filling data from the associated data and adding it to the correction matrix; for data to be corrected, obtaining relevant data from the associated data and inputting it into the analysis model, determining the correction value, and adding the correction value to the correction matrix. The analysis model is trained using federated learning. Provide data processing services based on the correction matrix of the participant and the local data of the participant, including: determining the trusted execution environment of the participant, sending the correction matrix to the participant so that the participant can fuse the local data with the correction matrix in the local trusted execution environment, and performing data processing based on the fused data.

Images