A data quality assessment method, computer device and storage medium

Abstract

The present invention discloses a data quality assessment method, a computer device and a storage medium, the method comprising: according to at least one data evaluation index, respectively assessing the quality of sample data in the participants and the initiator in federated learning, and obtaining local data assessment results corresponding to the participants and the initiator respectively; determining the federated data assessment results between the participants and the initiator according to the sample data influence between the participants and the initiator; determining the sample data quality assessment results corresponding to the participants and the initiator respectively according to the local data assessment results corresponding to the participants and the initiator respectively, and the federated data assessment results. The technical solution of the embodiment of the present invention provides a data quality assessment method suitable for federated learning, which can realize objective and accurate assessment of sample data in a federated learning environment.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a data quality assessment method, a computer device, and a storage medium. Background Art

[0002] With the development and large-scale application of artificial intelligence technology and big data technology, enterprises and government organizations are paying more and more attention to the protection of data privacy, making it difficult for huge amounts of data to form valuable circulation and exchange among each other.

[0003] At present, federated learning, as a new type of artificial intelligence technology, can conduct machine learning training involving multiple parties while ensuring data privacy and security. Federated learning mainly includes horizontal federated learning and vertical federated learning. Vertical federated learning aggregates the data features of multiple participants and ensures data security through homomorphic encryption and other methods. If the data quality of the participants cannot be properly evaluated before federated learning, the effect of federated learning cannot be guaranteed, and the overall efficiency of federated learning is usually lower than that of traditional machine learning, and the cost of relearning is huge. Therefore, it is particularly important to conduct a quality assessment of the data participating in federated learning before federated learning.

[0004] However, the current federated learning environment lacks a complete data quality assessment method, which affects the application and popularization of federated learning technology and restricts the development of cross-industry and cross-field artificial intelligence technology and industry. Summary of the invention

[0005] The embodiments of the present invention provide a data quality assessment method, a computer device, and a storage medium, which can objectively and accurately evaluate sample data in a federated learning environment.

[0006] In a first aspect, an embodiment of the present invention provides a data quality assessment method, the method comprising:

[0007] According to at least one data evaluation indicator, quality evaluation is performed on sample data of participants and initiators in federated learning respectively, and local data evaluation results corresponding to the participants and initiators are obtained respectively;

[0008] Determine the federated data evaluation result between the participant and the initiator according to the sample data impact between the participant and the initiator;

[0009] According to the local data evaluation results corresponding to the participant and the initiator respectively, and the federal data evaluation results, the sample data quality evaluation results corresponding to the participant and the initiator respectively are determined.

[0010] In a second aspect, an embodiment of the present invention further provides a computer device, including a processor and a memory, wherein the memory is used to store instructions, and when the instructions are executed, the processor performs the following operations:

[0011] According to at least one data evaluation indicator, quality evaluation is performed on sample data of participants and initiators in federated learning respectively, and local data evaluation results corresponding to the participants and initiators are obtained respectively;

[0012] Determine the federated data evaluation result between the participant and the initiator according to the sample data impact between the participant and the initiator;

[0013] According to the local data evaluation results corresponding to the participant and the initiator respectively, and the federal data evaluation results, the sample data quality evaluation results corresponding to the participant and the initiator respectively are determined.

[0014] In a third aspect, an embodiment of the present invention further provides a storage medium, the storage medium is used to store instructions, and the instructions are used to execute:

[0015] According to at least one data evaluation indicator, quality evaluation is performed on sample data of participants and initiators in federated learning respectively, and local data evaluation results corresponding to the participants and initiators are obtained respectively;

[0016] Determine the federated data evaluation result between the participant and the initiator according to the sample data impact between the participant and the initiator;

[0017] According to the local data evaluation results corresponding to the participant and the initiator respectively, and the federal data evaluation results, the sample data quality evaluation results corresponding to the participant and the initiator respectively are determined.

[0018] The embodiment of the present invention performs quality evaluation on sample data in participants and initiators in federated learning according to at least one data evaluation indicator, obtains local data evaluation results corresponding to the participants and initiators respectively, determines the federated data evaluation results between the participants and the initiators according to the influence of the sample data between the participants and the initiators, and determines the technical means of quality evaluation results of sample data corresponding to the participants and the initiators respectively according to the local data evaluation results corresponding to the participants and the initiators respectively and the federated data evaluation results, thereby providing a new data quality evaluation method suitable for a federated learning environment, which can ensure the accuracy of the sample data quality evaluation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1A flowchart of a data quality assessment method provided in Example 1 of the present invention;

[0020] Figure 2a A flowchart of a data quality assessment method provided in Embodiment 2 of the present invention;

[0021] Figure 2b A schematic diagram of a scenario applicable to a data quality assessment method provided in Embodiment 2 of the present invention;

[0022] Figure 3 A schematic diagram of the structure of a data quality assessment device provided in Embodiment 3 of the present invention;

[0023] Figure 4 A schematic diagram of the structure of a computer device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0025] It should be mentioned before discussing the exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations (or steps) as sequential processes, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0026] The term "federated learning" used in this article refers to federated machine learning, which is a machine learning framework that can effectively help multiple institutions use data and perform machine learning modeling while meeting the requirements of user privacy protection, data security, and government regulations.

[0027] The term "data evaluation index" used in this article may be an evaluation index artificially set in advance based on the quality requirements for sample data.

[0028] To facilitate understanding, the main inventive concepts of the embodiments of the present invention are briefly described.

[0029] Federated learning mainly includes horizontal federated learning and vertical federated learning. Vertical federated learning aggregates the data features of multiple participants and ensures data security through homomorphic encryption and other methods. If the data quality of the participants cannot be properly evaluated before federated learning, the effect of federated learning cannot be guaranteed, and the overall efficiency of federated learning is usually lower than that of traditional machine learning, and the cost of relearning is huge. Therefore, it is particularly important to conduct quality assessment of the data participating in federated learning before federated learning. However, the current lack of a complete data quality assessment method in the federated learning environment has affected the application and popularization of federated learning technology and restricted the development of cross-industry and cross-field artificial intelligence technology and industry.

[0030] In view of the problem that the prior art lacks an effective data quality assessment method for a federated learning environment, the inventors considered whether a method could be used to accurately assess the quality of sample data corresponding to federated learning, so as to mitigate the impact of poor quality data on learning results when data is not shared.

[0031] Based on the above considerations, the inventor creatively proposed a technical means to simultaneously consider the quality assessment of local sample data and federated learning data in the initiator and the participants, and to use comprehensive scores in multiple dimensions to measure data quality, thereby providing a new data quality assessment method suitable for a federated learning environment, which can ensure the accuracy of the sample data quality assessment results.

[0032] Embodiment 1

[0033] Figure 1 This is a flow chart of a data quality assessment method provided in the first embodiment of the present invention. The embodiment of the present invention is applicable to the case of performing quality assessment on data participating in federated learning. The method can be performed by a data quality assessment device provided in the embodiment of the present invention. The device can be implemented in software and / or hardware and can generally be integrated in a computer device. Figure 1 As shown, the method of the embodiment of the present invention specifically includes:

[0034] Step 110: According to at least one data evaluation indicator, quality evaluation is performed on sample data in the participants and the initiator in the federated learning, and local data evaluation results corresponding to the participants and the initiator are obtained.

[0035] In this embodiment, the initiator may be an institution or enterprise that initiates federated learning, and the participant may be an institution or enterprise that the initiator hopes to cooperate with. Prior to this step, the initiator and the participant respectively prepare sample data to be learned.

[0036] In this step, the quality of the sample data in the participants and the initiator of the federated learning can be evaluated according to at least one data evaluation index, wherein the data evaluation index may include data integrity and data accuracy.

[0037] In a specific embodiment, the number of missing values ​​included in the sample data of the participating parties and the initiator can be counted respectively, and the data integrity of each sample data can be scored according to the statistical results; then the number of abnormal values ​​in each sample data can be counted respectively, and the data accuracy of each sample data can be scored according to the statistical results; finally, the above two scores corresponding to each sample data are added to obtain the local data evaluation results corresponding to the participating parties and the local data evaluation results corresponding to the initiator.

[0038] Step 120: Determine the federated data evaluation result between the participant and the initiator according to the sample data impact between the participant and the initiator.

[0039] In this embodiment, in order to measure whether the sample data of the participants has an improvement effect on the overall data quality of federated learning, an implementation method is also provided for determining the federated data evaluation results between the participants and the initiator based on the impact of the sample data between the participants and the initiator.

[0040] In this step, optionally, a federated data evaluation result between the participant and the initiator may be determined based on the correlation between the sample data in the participant and the initiator, and the federated data evaluation result may be a score used to characterize the quality of the federated data.

[0041] In a specific embodiment, the correlation between the sample data in the participant and the initiator can be determined based on the data category to which the sample data in the participant belongs and the data category to which the sample data in the initiator belongs. Specifically, if the correlation between the sample data in the participant and the initiator is higher, it can be confirmed that the sample data in the participant and the initiator more satisfies the training requirements of the set business scenario, that is, the federated data evaluation result between the participant and the initiator is better.

[0042] In this embodiment, step 110 and step 120 may be executed in series or in parallel, and this embodiment does not limit the specific execution order.

[0043] Step 130: Determine the sample data quality assessment results corresponding to the participant and the initiator respectively according to the local data assessment results corresponding to the participant and the initiator respectively, and the federal data assessment result.

[0044] In this embodiment, optionally, the local data evaluation result corresponding to the participant can be added to the federal data evaluation result to obtain the final sample data quality evaluation result corresponding to the participant; the local data evaluation result corresponding to the initiator can be added to the federal data evaluation result to obtain the final sample data quality evaluation result corresponding to the initiator.

[0045] In this embodiment, under the premise of protecting the data privacy of the participants and the initiator, a two-level evaluation system of local data evaluation and federal data evaluation is established. At the same time, the local data evaluation results and the federal data evaluation results are combined to determine the final sample data quality evaluation results corresponding to the participants and the initiator respectively, providing a new data quality evaluation method suitable for a federated learning environment. On the one hand, it can ensure the accuracy of the sample data quality evaluation results; on the other hand, it can reduce the impact of poor quality data on the training model effect due to non-sharing of data, thereby promoting the application and popularization of federated learning technology.

[0046] The embodiment of the present invention performs quality evaluation on sample data in participants and initiators in federated learning according to at least one data evaluation indicator, obtains local data evaluation results corresponding to the participants and initiators respectively, determines the federated data evaluation results between the participants and the initiators according to the influence of the sample data between the participants and the initiators, and determines the technical means of quality evaluation results of sample data corresponding to the participants and the initiators respectively according to the local data evaluation results corresponding to the participants and the initiators respectively and the federated data evaluation results, thereby providing a new data quality evaluation method suitable for a federated learning environment, which can ensure the accuracy of the sample data quality evaluation results.

[0047] Embodiment 2

[0048] This embodiment is a further refinement of the above-mentioned embodiment 1. The explanation of terms that are the same or corresponding to those in the above-mentioned embodiment will not be repeated in this embodiment. Figure 2a Flow chart of a data quality assessment method provided in Embodiment 2 of the present invention. In this embodiment, the technical solution of this embodiment can be combined with one or more methods in the solutions of the above embodiments. In this embodiment, Figure 2a As shown, the method provided by the embodiment of the present invention may also include:

[0049] Step 210: According to at least one data evaluation indicator, quality evaluation is performed on sample data in the participants and the initiator in the federated learning, and local data evaluation results corresponding to the participants and the initiator are obtained.

[0050] In one implementation of the embodiment of the present invention, the data evaluation index may include: data duplication, data missing, data anomaly, and data singleness. According to at least one data evaluation index, the quality of the sample data of the participants in the federated learning is evaluated to obtain the local data evaluation result corresponding to the participant, including:

[0051] Step 211: Calculate the duplicate data scoring results, missing data scoring results, abnormal data scoring results, and single data scoring results corresponding to the participant according to the data evaluation indicators and the sample data of the participant;

[0052] In this step, the sample data of the participants can be deduplicated, and then the ratio λ of the number of samples after deduplication to the number of original samples is calculated. Assuming that the score result of duplicate data is a, then: a=round(λ,1).

[0053] In this step, the number of samples with missing features or a value type of "NULL" in the sample data and the ratio σ to the total number of samples can be counted. Assuming that the missing data scoring result is b, then: b=1-σ.

[0054] In this step, the number of samples with features belonging to outliers in the sample data and the ratio γ to the total number of samples can be counted. Assuming that the score result of the outlier data is c, then: c=round(1-γ, 1).

[0055] For continuous sample data, IQR is defined as the difference between the upper quartile and the lower quartile, and t is the preset threshold. Then, values ​​exceeding the upper limit (upper quartile + t*IQR) or less than the lower limit (lower quartile - t*IQR) can be defined as outliers. For discrete sample data, if the sample data is of coded type, values ​​outside the upper and lower limits of the coded type are defined as outliers.

[0056] In this step, the standard deviation of the sample data corresponding to different data categories can be calculated based on the sample data of the participants; the single indicator evaluation results corresponding to each data category can be determined based on the size relationship between the standard deviation of each sample data and the preset threshold, and the average of each single indicator evaluation result is taken to obtain the single data scoring result corresponding to the participant.

[0057] Among them, the preset threshold value can be 10e-8. Specifically, if the standard deviation of the sample data is less than the preset threshold value, the single indicator evaluation result corresponding to the standard deviation of the sample data can be set to 0; conversely, if the standard deviation of the sample data is greater than or equal to the preset threshold value, the single indicator evaluation result corresponding to the standard deviation of the sample data can be set to 1.

[0058] Step 212: Accumulate the duplicate data scoring results, missing data scoring results, abnormal data scoring results, and single data scoring results corresponding to the participant to obtain a local data evaluation result corresponding to the participant.

[0059] In this embodiment, the method for determining the local data evaluation result of the initiator is the same as that of the participants. The specific process can be found in the above steps 211-212, which will not be described in detail in this embodiment.

[0060] In this embodiment, optionally, if the local data evaluation result corresponding to the participant is lower than a set threshold value, it can be said that the sample data quality of the participant is poor, and the initiator can refuse to conduct federated learning with the participant.

[0061] In this embodiment, in addition to the above-mentioned data evaluation indicators, other indicators such as data validity, timeliness, consistency, significance test based on P value and multicollinearity VIF (variance inflation factor) test can also be considered to evaluate the local data quality.

[0062] Step 220: Calculate the sample data scoring result between the participant and the initiator according to the overlap ratio of the sample data between the participant and the initiator.

[0063] Before this step, in order to improve the accuracy of the federated data evaluation results, the privacy-preserving set intersection technology can be used to align the sample data of the participants and the sample data of the initiator.

[0064] In this step, assuming that the sample data of the initiator is A, the sample data of the participant is B, the overlap ratio of the sample data of the participant and the initiator is |A∩B| / |B|, τ is the preset threshold, and the score result of the sample data between the participant and the initiator is d, then:

[0065]

[0066] Step 230: Calculate the data information volume score between the participant and the initiator based on the labels and quantity of the sample data in the participant and the initiator.

[0067] In this step, the sample data in the participants and the initiator can be first divided into multiple sample groups, and then the sample data in each sample group is divided according to the label specified by the initiator, and the number of samples corresponding to each label is counted. Finally, according to the number of samples corresponding to each label and the number of samples corresponding to each sample group, the data information value (IV) score result between the participants and the initiator is calculated.

[0068] In a specific embodiment, it is assumed that the sample data of the participants and the initiator are divided into i sample groups, y i is the number of samples belonging to label 1 in the group, y s is the total number of samples belonging to label 1, n i is the number of samples belonging to label 2 in the group, n s is the total number of samples belonging to label 2, the IV value between the participant and the initiator can be calculated by the following formula:

[0069]

[0070] After calculating the IV value between the participant and the initiator, the IV score result e between the participant and the initiator can be calculated by the following formula:

[0071]

[0072] Step 240: Calculate the data correlation score between the participant and the initiator based on the data features of the sample data in the participant and the initiator.

[0073] In this step, it is assumed that the data characteristics of the sample data in the participant are α i , the data characteristics of the sample data in the initiator are β i , m is the number of samples included in the participant and the initiator respectively, then the linear correlation coefficient Corr between the participant and the initiator can be calculated by the following formula:

[0074]

[0075] In this step, after calculating the linear correlation coefficient Corr, the data correlation score result f between the participant and the initiator can be calculated by the following formula:

[0076]

[0077] Step 250: Accumulate the sample data scoring results, data information volume scoring results, and data relevance scoring results between the participant and the initiator to obtain the federated data evaluation result.

[0078] Step 260: Calculate the sample data quality assessment results corresponding to the participant and the initiator respectively based on the local data assessment results corresponding to the participant and the initiator respectively, the federal data assessment results, and a preset weighted ratio.

[0079] In this step, assuming that the preset weighted ratio is w1:w2, the local data evaluation result corresponding to the participant is p, the federal data evaluation result is q, and the sample data quality evaluation result corresponding to the participant is u, then: u=w1*p+w2*q.

[0080] Preferably, w1 can be set to 40%, and w2 can be set to 60%. The specific values ​​are preset according to actual conditions, and this embodiment does not limit this.

[0081] In this embodiment, the calculation method of the sample data quality evaluation result of the initiator is the same as that of the participating parties, and this embodiment will not be repeated.

[0082] In this embodiment, optionally, after determining the sample data quality assessment results corresponding to the participating party and the initiator respectively, it also includes: judging whether the sample data quality assessment result is less than a preset assessment threshold; if so, optimizing the sample data matching the sample data quality assessment result.

[0083] Optionally, the missing values, abnormal values ​​and duplicate values ​​in the sample data that match the sample data quality assessment results can be optimized. The advantage of this is that by optimizing the sample data with lower quality assessment results, the impact of poor quality data on the training model effect can be reduced.

[0084] In this embodiment, the integrity and accuracy of the sample data are evaluated from multiple dimensions such as data duplication, missing, abnormal and single situations, and local data evaluation results corresponding to the participants and the initiators are obtained; then, the influence of the sample data between the participants and the initiators is considered, and the federal data evaluation results are determined from dimensions such as data overlap, amount of information and linear correlation. Finally, the local data evaluation results are combined with the federal data evaluation results to finally determine the sample data quality evaluation results corresponding to the participants and the initiators, so as to achieve objective and accurate evaluation of the sample data in the federated learning environment.

[0085] The embodiment of the present invention performs quality evaluation on sample data of participants and initiators in federated learning according to data evaluation indicators to obtain local data evaluation results corresponding to the participants and initiators respectively, calculates sample data scoring results according to the overlap ratio of sample data in the participants and initiators, calculates data information volume scoring results according to labels and quantities of sample data in the participants and initiators, calculates data correlation scoring results according to data features of sample data in the participants and initiators, accumulates sample data scoring results, data information volume scoring results, and data correlation scoring results to obtain federated data evaluation results, and finally calculates technical means for sample data quality evaluation results corresponding to the participants and initiators respectively according to local data evaluation results corresponding to the participants and initiators, federated data evaluation results, and preset weighted ratios, thereby providing a new data quality evaluation method suitable for a federated learning environment, which can ensure the accuracy of sample data quality evaluation results.

[0086] In order to better introduce the technical solution provided by the embodiment of the present invention, Figure 2b This is a schematic diagram of a scenario applicable to a data quality assessment method in this embodiment. The embodiment of the present invention can refer to the following implementation methods:

[0087] In this implementation case, it is assumed that insurance company A has user asset valuation data, sports bracelet company B has user heart rate data, blood pressure data, exercise volume data and sports accident data. Insurance company A now hopes to cooperate with sports bracelet company B to assess the user's risk level. Insurance company A can be used as the initiator of federated learning, and sports bracelet company B can be used as a participant in federated learning. Among them, users can be divided into high-risk users and low-risk users according to the target training model (binary classification model).

[0088] exist Figure 2bIn the scenario shown, the client corresponding to the initiator is FL-A, and the client corresponding to the participant is FL-B, wherein FL-A includes the user asset valuation data sample Dataset-A and the sample tag Tag. Dataset-A has data of 2,000 users, and the Tag marks whether the user is a high-risk user, with a value of 0 or 1. FL-B only has sample data Dataset-B, which is the user's heart rate data, blood pressure data, exercise data, and whether there has been an accidental sports accident. There are 3,000 users' data in Dataset-B. FL-Server-C is a federated learning server that provides parameter services to the outside world. FL-Server-C, FL-A, and FL-B communicate through a network, which can be an Ethernet or an IB network (InfiniBand network). This embodiment takes vertical federated learning as an example. The participants and the initiator of the training cannot transmit original data to each other, and can only transmit encrypted data. In this implementation, the target model can be any model in actual application, and this embodiment does not limit the framework or structure of the target model.

[0089] like Figure 2b As shown, the specific implementation steps of the data quality assessment method may include:

[0090] Step 271, federated learning preparation: FL-A and FL-B prepare datasets Dataset-A and Dataset-B respectively, FL-A initiates a federated learning task, and FL-B joins the federated learning task.

[0091] Step 272, FL-A and FL-B respectively perform local data quality assessment, and calculate duplicate data scores, missing data scores, abnormal data scores, and single data scores. The specific process is as follows:

[0092] Step 2721, assuming that Dataset-A includes 2000 samples, of which there are 87 repeated samples. The repeated data score is calculated according to the repeated value calculation formula round((2000-87) / 2000, 1) as = 0.9. Dataset-B includes 3000 samples, of which there are 337 repeated samples. The repeated data score is calculated according to the repeated value calculation formula round((3000-645) / 3000, 1) as 0.8.

[0093] Step 2723, calculate the missing data score, abnormal data score and single data score for the sample data in Dataset-A and Dataset-B respectively: Assuming that the user asset valuation data column in Dataset-A has 39 null missing values, the missing data score calculated according to the formula is (2000-39) / 2000=0.9805; the user risk level data column in Dataset-A has 12 null missing values, and the missing data score calculated according to the formula is (2000-12) / 2000=0.99 4; There are 72 null missing values ​​in the user heart rate data column in Dataset-B. According to the formula, the missing data score is (3000-72) / 3000=0.976; There are 75 null missing values ​​in the user blood pressure data column in Dataset-B. According to the formula, the missing data score is (3000-75) / 3000=0.975; There are 157 null missing values ​​in the user exercise data column in Dataset-B. The missing data score is (3000-157) / 3000=0.947. There are 394 null missing values ​​in the user sports accident data column in Dataset-B. The missing data score is (3000-394) / 3000=0.868.

[0094] In this embodiment, for continuous sample data, IQR is defined as the difference between the upper quartile and the lower quartile, t is a preset threshold, and values ​​exceeding the upper limit (upper quartile + t*IQR) or less than the lower limit (lower quartile - t*IQR) can be defined as outliers. For discrete sample data, if the sample data is of coded type, values ​​outside the upper and lower limits of the coded type are defined as outliers, where t can be 1.5.

[0095] In this embodiment, the specific number of outliers is known. According to the formula round((2000-658 / 2000), 1), the outlier score of the user asset valuation data column in Dataset-A can be calculated to be 0.7; according to the formula round((2000-426 / 2000), 1), the outlier score of the user risk level data column in Dataset-A can be calculated to be 0.8; according to the formula round((3000-665 / 3000), 1), the outlier score of the user risk level data column in Dataset-B can be calculated to be 0.8. The abnormal data score of the rate data column is 0.8; according to the formula round((3000-649 / 3000), 1), the abnormal data score of the user blood pressure data column in Dataset-B is 0.8; according to the formula round((3000-232 / 3000), 1), the abnormal data score of the user exercise data column in Dataset-B is 0.9; according to the formula round((3000-0 / 3000), 1), the abnormal data score of the user sports accident data column in Dataset-B is 1.

[0096] Calculate the single data score: If the standard deviation of the sample data is less than the preset threshold, the single indicator evaluation result is 0, otherwise it is 1. In this embodiment, the standard deviation of the user asset valuation data column in Dataset-A is calculated according to the standard deviation formula to be 1836, which is greater than 10e-8, and the single data score is 1; the standard deviation of the user risk level data column in Dataset-A is 82, which is greater than 10e-8, and the single data score is 1; the standard deviation of the user heart rate data column in Dataset-B is 178, which is greater than 10e-8, and the single data score is 1; the standard deviation of the user blood pressure data column in Dataset-B is 317, which is greater than 10e-8, and the single data score is 1; the standard deviation of the user exercise data column in Dataset- is 9×10e-9, which is less than 10e-8, and the single data score is 1. The standard deviation formula can be:

[0097] s=sqrt(((x1-x)^2+(x2-x)^2+…(x n -x)^2) / (n-1))

[0098] Based on the scores of the above indicators, the local data score can be calculated. The local data score can be calculated by averaging the duplicate data score, missing data score, abnormal data score and single data score. Through the above calculation process, the initiator's local data score can be obtained as follows:

[0099] 0.9+(0.9805+0.994) / 2+(0.7+0.8) / 2+(1+1) / 2=3.637

[0100] The local data scores of the participants are:

[0101] 0.8+(0.976+0.975+0.947+0.868) / 4+(0.8+0.8+0.9+1) / 4+(1+1+1+0) / 4=3.366 In this embodiment, the total score of the local data score can be set to 100, so the local data score of the initiator can be 90.925, and the local data score of the participant can be 84.15. Since the local scores of the initiator and the participant are both excellent, FL-A and FL-B can accept each other for federated learning.

[0102] Step 273, conduct a federated data quality assessment. Before federated learning, a comprehensive quality assessment of multi-party data is required. The federated data quality assessment is used to determine whether the participants have improved the overall data quality. The specific approach is to first perform sample alignment on the data, and then calculate the multi-dimensional score to evaluate the data quality in the federated environment. In this embodiment, the privacy-preserving set intersection technology can be used for sample alignment, and then the multi-party data quality score in the federated environment is calculated: sample data score, IV value score and data relevance score. The federated data assessment result is obtained by adding the above three scores. In this embodiment, the total score of the federated data score is 100, and the sample data score, IV value score and data relevance score each account for one-third. The sample data score, IV value score and data relevance score initially calculated are all full marks of 2, which need to be multiplied by a proportional coefficient of 50 / 3.

[0103] The sample data score can be calculated by the following formula, τ = 0.8:

[0104]

[0105] In this embodiment, |A| represents the sample size in Dataset-A in FL-A, |B| represents the sample size in Dataset-B in FL-B, and |A∩B| is the number of identical users in Dateset-A and Dataset-B. Specifically, |A∩B| is 1956, and |B| is 3000. Since |A∩B| / |B| is 0.652, which is less than 0.8, it belongs to the first case. According to the above formula, d is calculated to be 1.624, which then needs to be multiplied by the total ratio coefficient 50 / 3. The final sample data score is 27.066.

[0106] FL-Server-C creates a key pair and sends the public key to FL-A and FL-B. FL-A performs Paillier encryption on each sample ID. i ]],[[1-y i]], and then send the plaintext ID to FL-B. FL-B performs feature binning and performs homomorphic addition summation on the ciphertext label values ​​corresponding to the ID in each bin:

[0107] [[∑y i ]]=∑[[y i ]],[[∑(1-y i )]]=∑[[1-y i ]]

[0108] Then the bins corresponding to each ID are sent to FL-Server-C. FL-Server-C decrypts and obtains ∑y i ,∑(1-y i ), and then calculate the IV value for each column of the data:

[0109]

[0110] where y i is the number of samples belonging to label 1 in the group, y s is the total number of samples belonging to label 1, n i is the number of samples belonging to label 2 in the group, n s is the total number of samples belonging to label 2. In this embodiment, the samples are divided into 4 groups, y s The total number of samples belonging to low-risk users is 1136, n s The total number of samples belonging to high-risk users is 820, and group 1 has a total of 431 samples. y1 is the number of low-risk users in group 1, which is 138. n1 is the number of high-risk users in group 1, which is 293. Group 2 has a total of 846 samples. y2 is the number of low-risk users in group 2, which is 708. n2 is the number of high-risk users in group 2, which is 138. Group 3 has a total of 511 samples. y3 is the number of low-risk users in group 3, which is 254. n3 is the number of high-risk users in group 3, which is 257. Group 4 has a total of 168 samples. y4 is the number of low-risk users in group 4, which is 36. n4 is the number of high-risk users in group 4, which is 132. The IV value can be calculated according to the following formula:

[0111]

[0112] After calculating the IV value between the participant and the initiator, the IV score result e between the participant and the initiator can be determined according to the following formula:

[0113]

[0114] Since the IV value is greater than 0.3, e is 2, and it needs to be multiplied by the total ratio coefficient 50 / 3. The final IV score is 33.333.

[0115] FL-Server-C creates a key pair and sends the public key to FL-A and FL-B. FL-B uses the formula Calculate Var(X) of each column, and encrypt it through Paillier to get [[Var(X)]] and send it to FL-A. FL-A calculates Var(Y), calculates [[Var(X)]]Var(Y) and sends it to FL-Server-C. FL-Server-C receives [[V a(r)X]](V a) and decrypts it to get Var(X)Var(Y). FL-B calculates Diff(X), where Then, [[Diff(X)]] is obtained through Paillier encryption and sent to FL-A. FL-A generates a random mask Rin(0,1), the vector inner product of [[ffDiX()]] and Diff(Y) is [[sum(x_i-\barx)(y_i-\bary)]], calculates R[[Di(ff)X]](Di)f and sends it to FL-Server-C. FL-Server-C receives R[[Di(ff)X]](Di)f, decrypts it to get RCov(X,Y), and then calculates Corr, where: Corr=RCov(X,Y) / (Var(X)Var(Y))^(1 / 2), and sends the result to FL-A. After FL-A receives RCorr(X,Y), it calculates Corr(X,Y). In this embodiment, the calculation result of Corr(X,Y) is 1.74, which needs to be multiplied by the proportional coefficient 50 / 3. The final Corr score (that is, the data correlation score) is 29.166.

[0116] In summary, the federated data score can be calculated based on the above indicator scores. The specific calculation method is: add the sample data score, IV score and Corr score. In this embodiment, the federated data score is 27.066+33.333+29.166=89.565.

[0117] Step 274, calculate the comprehensive score of local data quality and federated data quality, where the local data quality score accounts for 40% and the federated data quality score accounts for 60%. In this embodiment, the comprehensive score of FL-A is 90.925×0.4+89.565×0.6=90.119; the comprehensive score of FL-B is 84.15×0.4+89.565×0.6=87.399. Based on this comprehensive score, the federated learning process can be optimized. If the score is too low, the sample data of the party with the lower score can be adjusted.

[0118] The method provided in the embodiment of the present invention provides a novel data quality assessment method suitable for a federated learning environment, which can ensure the accuracy of the sample data quality assessment results.

[0119] Embodiment 3

[0120] Figure 3 This is a schematic diagram of the structure of a data quality assessment device provided in Embodiment 3 of the present invention. The device can be implemented in software and / or hardware, and can generally be integrated into a computer device. Figure 3 As shown, the apparatus includes: a local evaluation module 310 , a federal evaluation module 320 and a sample quality evaluation module 330 .

[0121] The local evaluation module 310 is used to perform quality evaluation on sample data of the participants and the initiator in the federated learning according to at least one data evaluation index, and obtain local data evaluation results corresponding to the participants and the initiator respectively;

[0122] A federation evaluation module 320, configured to determine a federation data evaluation result between the participant and the initiator according to the sample data impact between the participant and the initiator;

[0123] The sample quality assessment module 330 is used to determine the sample data quality assessment results corresponding to the participant and the initiator respectively according to the local data assessment results corresponding to the participant and the initiator respectively and the federated data assessment results.

[0124] The embodiment of the present invention performs quality evaluation on sample data in participants and initiators in federated learning according to at least one data evaluation indicator, obtains local data evaluation results corresponding to the participants and initiators respectively, determines the federated data evaluation results between the participants and the initiators according to the influence of the sample data between the participants and the initiators, and determines the technical means of quality evaluation results of sample data corresponding to the participants and the initiators respectively according to the local data evaluation results corresponding to the participants and the initiators respectively and the federated data evaluation results, thereby providing a new data quality evaluation method suitable for a federated learning environment, which can ensure the accuracy of the sample data quality evaluation results.

[0125] Based on the above embodiments, the local evaluation module 310 may include:

[0126] An indicator scoring calculation unit is used to calculate the duplicate data scoring results, missing data scoring results, abnormal data scoring results, and single data scoring results corresponding to the participant according to each of the data evaluation indicators and the sample data of the participant; the data evaluation indicators include: data duplication, data missing, data abnormality, and data singleness;

[0127] An indicator scoring accumulation unit, used to accumulate the duplicate data scoring results, missing data scoring results, abnormal data scoring results, and single data scoring results corresponding to the participant to obtain a local data evaluation result corresponding to the participant;

[0128] A standard deviation calculation unit, used to calculate the standard deviation of sample data corresponding to different data categories according to the sample data of the participant;

[0129] A single result processing unit is used to determine the single indicator evaluation results corresponding to each of the data categories according to the size relationship between the standard deviation of each of the sample data and a preset threshold, and to take the average of each of the single indicator evaluation results to obtain a single data scoring result corresponding to the participant.

[0130] Optionally, the federation assessment module 320 may include:

[0131] A sample data scoring calculation unit, used to calculate the sample data scoring result between the participant and the initiator according to the overlap ratio of the sample data between the participant and the initiator;

[0132] An information volume scoring calculation unit, used to calculate the data information volume scoring result between the participant and the initiator according to the labels and quantities of the sample data in the participant and the initiator;

[0133] A correlation score calculation unit, used to calculate the data correlation score result between the participant and the initiator according to the data characteristics of the sample data in the participant and the initiator;

[0134] The federation evaluation result determination unit is used to accumulate the sample data scoring results, data information volume scoring results, and data correlation scoring results between the participants and the initiator to obtain the federation data evaluation result.

[0135] Optionally, the sample quality assessment module 330 may include:

[0136] A weighted processing unit is used to calculate the sample data quality assessment results corresponding to the participant and the initiator respectively according to the local data assessment results corresponding to the participant and the initiator respectively, the federal data assessment results, and a preset weighting ratio.

[0137] Optionally, the data quality assessment device may further include:

[0138] An evaluation result judging unit, used to judge whether the sample data quality evaluation result is less than a preset evaluation threshold;

[0139] The data optimization unit is used to optimize the sample data matching the sample data quality evaluation result when the sample data quality evaluation result is less than a preset evaluation threshold.

[0140] The above-mentioned data quality assessment device can execute the data quality assessment method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the data quality assessment method.

[0141] Embodiment 4

[0142] Figure 4 A schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention is shown in FIG. Figure 4 As shown, the computer device includes a processor 410, a memory 420, an input device 430, and an output device 440; the number of processors 410 in the computer device can be one or more. Figure 4 A processor 410 is taken as an example; the processor 410, memory 420, input device 430 and output device 440 in the computer device can be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0143] The memory 420, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to a data quality assessment method in an embodiment of the present invention (for example, a local assessment module 310, a federal assessment module 320 and a sample quality assessment module 330 in a data quality assessment device). The processor 410 executes various functional applications and data processing of the computer device by running the software programs, instructions and modules stored in the memory 420, that is, implements the above-mentioned data quality assessment method. That is, when the program is executed by the processor, it implements:

[0144] According to at least one data evaluation indicator, quality evaluation is performed on sample data of participants and initiators in federated learning respectively, and local data evaluation results corresponding to the participants and initiators are obtained respectively;

[0145] Determine the federated data evaluation result between the participant and the initiator according to the sample data impact between the participant and the initiator;

[0146] According to the local data evaluation results corresponding to the participant and the initiator respectively, and the federal data evaluation results, the sample data quality evaluation results corresponding to the participant and the initiator respectively are determined.

[0147] The memory 420 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 420 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 420 may further include a memory remotely arranged relative to the processor 410, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0148] The input device 430 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the computer device, and may include a keyboard and a mouse, etc. The output device 440 may include a display device such as a display screen.

[0149] Based on the above embodiments, the data evaluation indicators include: data duplication, data missing, data anomaly and data singleness;

[0150] The processor 410 is configured to perform quality assessment on sample data of a participant in federated learning according to at least one data evaluation indicator in the following manner to obtain a local data evaluation result corresponding to the participant:

[0151] According to each of the data evaluation indicators and the sample data of the participant, respectively calculate the duplicate data scoring results, missing data scoring results, abnormal data scoring results, and single data scoring results corresponding to the participant;

[0152] The duplicate data scoring results, missing data scoring results, abnormal data scoring results, and single data scoring results corresponding to the participant are accumulated to obtain the local data evaluation results corresponding to the participant.

[0153] The processor 410 is configured to determine the federated data evaluation result between the participant and the initiator according to the sample data impact between the participant and the initiator in the following manner:

[0154] Calculate the sample data scoring result between the participant and the initiator according to the overlap ratio of the sample data between the participant and the initiator;

[0155] Calculate the data information volume score between the participant and the initiator according to the labels and quantity of the sample data in the participant and the initiator;

[0156] Calculate the data correlation score between the participant and the initiator according to the data characteristics of the sample data in the participant and the initiator;

[0157] The sample data scoring results, data information volume scoring results, and data relevance scoring results between the participants and the initiator are accumulated to obtain the federated data evaluation result.

[0158] The processor 410 is configured to determine the sample data quality evaluation results corresponding to the participant and the initiator respectively according to the local data evaluation results corresponding to the participant and the initiator respectively and the federated data evaluation results in the following manner:

[0159] According to the local data evaluation results corresponding to the participant and the initiator respectively, the federal data evaluation results, and a preset weighting ratio, the sample data quality evaluation results corresponding to the participant and the initiator respectively are calculated.

[0160] After determining the sample data quality assessment results corresponding to the participant and the initiator, the processor 410 further performs the following operations:

[0161] Determine whether the sample data quality assessment result is less than a preset assessment threshold;

[0162] If so, the sample data that matches the sample data quality assessment result is optimized.

[0163] The processor 410 is configured to calculate a single data scoring result corresponding to the participant according to the data evaluation index and the sample data of the participant in the following manner:

[0164] Calculate the standard deviation of the sample data corresponding to different data categories according to the sample data of the participants;

[0165] According to the size relationship between the standard deviation of each sample data and the preset threshold, the single indicator evaluation results corresponding to each data category are determined, and the average of each single indicator evaluation result is taken to obtain the single data scoring result corresponding to the participant.

[0166] Embodiment 5

[0167] Embodiment 5 of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in any embodiment of the present invention is implemented. Of course, a computer-readable storage medium provided by an embodiment of the present invention can perform related operations in a data quality assessment method provided by any embodiment of the present invention. That is, when the program is executed by a processor, it implements:

[0168] According to at least one data evaluation indicator, quality evaluation is performed on sample data of participants and initiators in federated learning respectively, and local data evaluation results corresponding to the participants and initiators are obtained respectively;

[0169] Determine the federated data evaluation result between the participant and the initiator according to the sample data impact between the participant and the initiator;

[0170] According to the local data evaluation results corresponding to the participant and the initiator respectively, and the federal data evaluation results, the sample data quality evaluation results corresponding to the participant and the initiator respectively are determined.

[0171] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0172] It is worth noting that in the above-mentioned embodiment of the data quality assessment device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0173] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data quality assessment method, characterized in that: include: According to at least one data evaluation indicator, quality evaluation is performed on sample data of participants and initiators in federated learning respectively, and local data evaluation results corresponding to the participants and initiators are obtained respectively; Determine the federated data evaluation result between the participant and the initiator according to the sample data impact between the participant and the initiator; Determine the sample data quality assessment results corresponding to the participant and the initiator respectively according to the local data assessment results corresponding to the participant and the initiator respectively and the federal data assessment result; According to the influence of the sample data between the participant and the initiator, the evaluation result of the federated data between the participant and the initiator is determined, including: calculating the sample data scoring result between the participant and the initiator according to the overlap ratio of the sample data in the participant and the initiator; calculating the data information amount scoring result between the participant and the initiator according to the labels and quantity of the sample data in the participant and the initiator; calculating the data correlation scoring result between the participant and the initiator according to the data characteristics of the sample data in the participant and the initiator; accumulating the sample data scoring result, data information amount scoring result, and data correlation scoring result between the participant and the initiator to obtain the federated data evaluation result; The data evaluation indicators include: data duplication, data missing, data anomaly and data singleness; According to at least one data evaluation indicator, the quality of the sample data of the participants in the federated learning is evaluated to obtain the local data evaluation results corresponding to the participants, including: According to each of the data evaluation indicators and the sample data of the participant, respectively calculate the duplicate data scoring results, missing data scoring results, abnormal data scoring results, and single data scoring results corresponding to the participant; The duplicate data scoring results, missing data scoring results, abnormal data scoring results, and single data scoring results corresponding to the participant are accumulated to obtain the local data evaluation results corresponding to the participant.

2. The method according to claim 1, characterized in that: Determine the sample data quality assessment results corresponding to the participant and the initiator respectively according to the local data assessment results corresponding to the participant and the initiator respectively, and the federal data assessment results, including: According to the local data evaluation results corresponding to the participant and the initiator respectively, the federal data evaluation results, and a preset weighting ratio, the sample data quality evaluation results corresponding to the participant and the initiator respectively are calculated.

3. The method according to claim 1, characterized in that After determining the sample data quality assessment results corresponding to the participants and the initiator, the method further includes: Determine whether the sample data quality assessment result is less than a preset assessment threshold; If so, the sample data that matches the sample data quality assessment result is optimized.

4. The method according to claim 1, characterized in that According to the data evaluation index and the sample data of the participant, a single data scoring result corresponding to the participant is calculated, including: Calculate the standard deviation of the sample data corresponding to different data categories according to the sample data of the participants; According to the size relationship between the standard deviation of each sample data and the preset threshold, the single indicator evaluation results corresponding to each data category are determined, and the average of each single indicator evaluation result is taken to obtain the single data scoring result corresponding to the participant.

5. A computer device comprising a processor and a memory, wherein the memory is used to store instructions, and when the instructions are executed, the processor performs the following operations: According to at least one data evaluation indicator, quality evaluation is performed on sample data of participants and initiators in federated learning respectively, and local data evaluation results corresponding to the participants and initiators are obtained respectively; Determine the federated data evaluation result between the participant and the initiator according to the sample data impact between the participant and the initiator; Determine the sample data quality assessment results corresponding to the participant and the initiator respectively according to the local data assessment results corresponding to the participant and the initiator respectively and the federal data assessment result; The processor is configured to determine the federated data evaluation result between the participant and the initiator according to the sample data impact between the participant and the initiator in the following manner: According to the overlap ratio of sample data in the participant and the initiator, the sample data scoring result between the participant and the initiator is calculated; according to the labels and quantity of the sample data in the participant and the initiator, the data information amount scoring result between the participant and the initiator is calculated; according to the data characteristics of the sample data in the participant and the initiator, the data correlation scoring result between the participant and the initiator is calculated; the sample data scoring result, data information amount scoring result, and data correlation scoring result between the participant and the initiator are accumulated to obtain the federated data evaluation result; The data evaluation indicators include: data duplication, data missing, data anomaly and data singleness; The processor is configured to perform quality assessment on sample data of a participant in federated learning according to at least one data evaluation indicator in the following manner to obtain a local data evaluation result corresponding to the participant: According to each of the data evaluation indicators and the sample data of the participant, respectively calculate the duplicate data scoring results, missing data scoring results, abnormal data scoring results, and single data scoring results corresponding to the participant; The duplicate data scoring results, missing data scoring results, abnormal data scoring results, and single data scoring results corresponding to the participant are accumulated to obtain the local data evaluation results corresponding to the participant.

6. The computer device according to claim 5, characterized in that The processor is configured to determine the sample data quality assessment results corresponding to the participant and the initiator respectively according to the local data assessment results corresponding to the participant and the initiator respectively and the federated data assessment results in the following manner: According to the local data evaluation results corresponding to the participant and the initiator respectively, the federal data evaluation results, and a preset weighting ratio, the sample data quality evaluation results corresponding to the participant and the initiator respectively are calculated.

7. The computer device according to claim 5, characterized in that After determining the sample data quality assessment results corresponding to the participant and the initiator respectively, the processor further performs the following operations: Determine whether the sample data quality assessment result is less than a preset assessment threshold; If so, the sample data that matches the sample data quality assessment result is optimized.

8. The computer device according to claim 5, characterized in that The processor is configured to calculate a single data scoring result corresponding to the participant according to the data evaluation index and the sample data of the participant in the following manner: Calculate the standard deviation of the sample data corresponding to different data categories according to the sample data of the participants; According to the size relationship between the standard deviation of each sample data and the preset threshold, the single indicator evaluation results corresponding to each data category are determined, and the average of each single indicator evaluation result is taken to obtain the single data scoring result corresponding to the participant.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a data quality assessment method as described in any one of claims 1 to 4 is implemented.

Images