A method, system, medium and computing device for power work order data quality management

Abstract

The present invention discloses a method, system, medium and computing device for quality management of electric power work order data in the field of electronic information technology, aiming to solve the problem in the prior art that poor data set quality leads to inaccurate prediction calculations. It includes acquiring and merging electric power work order data from different data sources to obtain a data set; calculating the data quality score of each record in the data set; calculating the overall quality score of the data set; judging whether the overall quality score is higher than the quality qualified threshold; filtering out unqualified records in the data set; deleting or repairing unqualified records in the data set to obtain a new data set; outputting the data set as a managed data set; the present invention is applicable to the quality management of electric power work order data, and can evaluate the data set and adjust the problem data until the overall quality meets the requirements, thereby improving the overall quality of the data set, avoiding the occurrence of unnecessary erroneous calculations, saving computing resources, and ensuring the accuracy of prediction calculations.

Description

Technical Field

[0001] The present invention relates to the field of electronic information technology, and in particular to a method, system, medium and computing device for managing the quality of electric power work order data. Background Art

[0002] With the continuous construction and deepening application of enterprise informatization, various business operations of the enterprise have been initially integrated with informatization. The quantity and types of business data in the information system have gradually increased, and the requirements for data quality have increased dramatically. When conducting relevant forecast calculations for power work orders, the quality of power work order information data will directly affect the actual business results.

[0003] However, due to the complex sources of power work order information data and uneven data quality, the quality of the data set obtained is poor, so when using it for prediction calculations, useful conclusions are often not drawn, and even wrong results are obtained. Therefore, how to manage data quality based on the characteristics of power work order information data, so as to timely discover errors and distortions in the data, and shield or repair them is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide a method, system, medium and computing device for managing the quality of electric power work order data to solve the problem that the current data set quality is poor and causes inaccurate prediction calculations.

[0005] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present invention provides a method for managing the quality of electric power work order data, comprising the following steps:

[0007] S1: Obtain and merge power work order data from different data sources to obtain a data set;

[0008] S2: Calculate the data quality score of each record in the data set according to the predefined field defect judgment rules and defect level definition;

[0009] S3: Calculate the overall quality score of the data set based on the data quality score of each record;

[0010] S4: Determine whether the overall quality score is higher than a preset quality threshold, if yes, execute S7, if no, execute S5;

[0011] S5: Filter out unqualified records in the data set based on the overall quality score and the data quality score of each record;

[0012] S6: Delete or repair unqualified records in the data set to obtain a new data set, and return to S2;

[0013] S7: Output the dataset as a governed dataset.

[0014] Furthermore, the calculation of the data quality score of each record in the data set according to the predefined field defect judgment rule and defect level definition includes:

[0015] Calculate the index score of each record according to the field defect judgment rule and defect level definition, wherein the index score includes a completeness score, an accuracy score and a consistency score;

[0016] According to the indicator score of each record, the data quality score of each record is calculated.

[0017] Furthermore, the calculation of the index score of each record according to the field defect judgment rule and defect level definition includes:

[0018] Find the defects that affect the indicator score in each record according to the field defect judgment rules;

[0019] According to the defect level definition, the defect is classified into defect levels, wherein the defect levels include minor, general, severe and fatal;

[0020] The indicator score of each record is calculated according to the following formula:

[0021]

[0022] In the formula, s represents the index score, o represents the number of minor defects, p represents the number of general defects, q represents the number of serious defects, r represents the number of fatal defects, i represents the deduction value of minor defects, j represents the deduction value of general defects, k represents the deduction value of serious defects, and l represents the deduction value of fatal defects.

[0023] Furthermore, the step of calculating the data quality score of each record according to the indicator score of each record includes:

[0024] The data quality score of each record is calculated according to the following formula:

[0025] t=a*x+b*y+c*z

[0026] In the formula, t represents the data quality score, a represents the completeness score, b represents the accuracy score, c represents the consistency score, x represents the completeness weight, y represents the accuracy weight, and z represents the consistency weight.

[0027] Furthermore, calculating the overall quality score of the data set according to the data quality score of each record includes:

[0028] The overall quality score of the dataset is calculated according to the following formula:

[0029] T=(t1+t2+t3……+tn) / n

[0030] In the formula, T represents the overall quality score of the data set, t1, t2, t3, ... tn represent the data quality scores of each record, and n represents the total number of records in the data set.

[0031] Furthermore, the filtering out of unqualified records in the data set according to the overall quality score and the data quality score of each record includes:

[0032] Calculate the qualified deviation value of each record based on the overall quality score and the data quality score of each record;

[0033] The qualifiedness of each record is determined according to the qualified deviation value of each record. If the qualified deviation value of a record is greater than zero, the record is determined to be an unqualified record, otherwise, the record is determined to be a qualified record.

[0034] Further, the calculation of the qualified deviation value of each record according to the overall quality score and the data quality score of each record includes:

[0035] The qualified deviation value of each record is calculated according to the following formula:

[0036] d=Tt-γ*δ

[0037] Where d represents the qualified deviation value, T represents the overall quality score, t represents the data quality score, γ represents the deviation coefficient, and δ represents the overall standard deviation of the data quality score of each record.

[0038] In a second aspect, the present invention provides a power work order data quality management system, comprising:

[0039] A data set acquisition module, used to acquire and merge power work order data from different data sources to obtain a data set;

[0040] The data quality calculation module is used to calculate the data quality score of each record in the data set according to the predefined field defect judgment rules and defect level definitions;

[0041] The overall quality calculation module is used to calculate the overall quality score of the data set based on the data quality score of each record;

[0042] The overall quality judgment module is used to judge whether the overall quality score is higher than the preset quality qualification threshold. If so, the data set output module is executed; if not, the data record screening module is executed;

[0043] The data record screening module is used to screen out unqualified records in the data set based on the overall quality score and the data quality score of each record;

[0044] The data record revision module is used to delete or repair unqualified records in the data set to obtain a new data set and return it to the overall quality calculation module;

[0045] The dataset output module is used to output the dataset as a managed dataset.

[0046] In a third aspect, the present invention provides a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any one of the methods described in the first aspect.

[0047] In a fourth aspect, the present invention provides a computing device, comprising:

[0048] One or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the methods described in the first aspect.

[0049] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0050] 1. The present invention first evaluates the overall quality of the data set to see whether it meets the quality requirements. If it does not meet the quality requirements, the problem data is quickly located and screened out, and the problem data is adjusted until the overall quality meets the requirements. In this way, the power work order information data is managed, the overall quality of the data set is improved, thereby avoiding unnecessary erroneous calculations, saving computing resources, and ensuring the accuracy of prediction calculations;

[0051] 2. The present invention calculates the data quality score of each record in the data set from the three aspects of completeness, accuracy and consistency, and then calculates the overall quality score of the data set based on the data quality score of each record. Its quality assessment method is scientific and objective, and can accurately reflect the data quality of the data set, thereby ensuring the reliability of quality management and having a better overall effect;

[0052] 3. The present invention calculates the qualified deviation value of each record in the data set, and then determines whether each record is qualified based on the qualified deviation value. It can quickly and conveniently screen out unqualified records in the data set, thereby improving the effect of data governance. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0054] Figure 1 It is a flow chart of a method for managing power work order data quality provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0056] Embodiment 1:

[0057] like Figure 1 As shown, an embodiment of the present invention provides a method for managing the quality of power work order data, comprising the following steps:

[0058] S1: Obtain and merge power work order data from different data sources to obtain a data set.

[0059] It should be noted that the merging rules are to first adjust the field format according to unified requirements, trim redundant fields, and splice missing fields to trim the power work order data, and then remove duplicates according to the work order number.

[0060] S2: Calculate the data quality score of each record in the data set based on the predefined field defect judgment rules and defect level definitions.

[0061] S21: Calculate the index score of each record according to the field defect judgment rule and the defect level definition, wherein the index score includes a completeness score, an accuracy score, and a consistency score.

[0062] S211: Find the defects that affect the index score in each record according to the field defect judgment rule.

[0063] Specifically, the field defect judgment rules in this example are as follows:

[0064] Fields Completeness accuracy consistency Work Order Number Not empty 8 digits - Work order initiation time Not empty yyyy-MM-ddhh:mm:ss - Work order completion time - yyyy-MM-ddhh:mm:ss Completion time is less than initiation time Username Not empty - - User age - [0,150] - User Gender - Male|Female - User electricity category Not empty Residents | Agriculture | Industry and Commerce - The area where the user is using electricity Not empty Areas included in the electricity consumption area list The electricity consumption area falls within the jurisdiction of the management agency User's management organization Not empty Institutions included in the list of management institutions - Power voltage level Not empty Voltage in the voltage level list - Power consumption agreement capacity Not empty (0,999999] - Power reliability requirements Not empty 99％|99.9％|99.99％ - Appeal Category Not empty Belongs to the category in the list of appeal categories - Content of the appeal Not empty - - Work Order Progress Not empty To be assigned | In process | Completed The completion time cannot be empty if it is completed. Data time Not empty Timestamp -

[0065] “-” means that the data in this field has no effect on this indicator.

[0066] S212: Classifying the defects into defect levels according to defect level definitions, wherein the defect levels include minor, general, severe, and fatal.

[0067] Specifically, the defect level definition in this example is as follows:

[0068]

[0069] “-” indicates that the field data has no corresponding defect level. A single field may have multiple defects belonging to the same defect level but different indicators.

[0070] S213: Calculate the index score of each record according to the following formula:

[0071]

[0072] In the formula, s represents the index score, o represents the number of minor defects, p represents the number of general defects, q represents the number of serious defects, r represents the number of fatal defects, i represents the deduction value of minor defects, j represents the deduction value of general defects, k represents the deduction value of serious defects, and l represents the deduction value of fatal defects.

[0073] It should be noted that when the index score s is less than 0, it is taken as 0; in this embodiment, the deduction points for minor defects, general defects, serious defects, and fatal defects are set to 3, 9, 27, and 81, respectively.

[0074] S22: Calculate the data quality score of each record according to the indicator score of each record.

[0075] In this embodiment, the data quality score of each record is calculated according to the following formula:

[0076] t=a*x+b*y+c*z

[0077] In the formula, t represents the data quality score, a represents the completeness score, b represents the accuracy score, c represents the consistency score, x represents the completeness weight, y represents the accuracy weight, and z represents the consistency weight.

[0078] It should be noted that the integrity weight, accuracy weight, and consistency weight can be estimated through subjective hierarchical analysis method, order relationship analysis method, objective entropy weight method, anti-entropy weight method, and subjective and objective comprehensive method.

[0079] Specifically, the integrity weight, accuracy weight and consistency weight in this embodiment are evaluated by subjective hierarchical analysis method, and the evaluation results are: the integrity index weight is 0.6333, the accuracy index weight is 0.1062, and the consistency index weight is 0.2605.

[0080] The evaluation process of AHP is as follows:

[0081] According to the following importance table:

[0082]

[0083]

[0084] Construct indicator relationship matrix:

[0085] Completeness accuracy consistency Completeness 1 5 3 accuracy 1 / 5 1 1 / 3 consistency 1 / 3 3 1

[0086] According to the matrix calculation, CI = 0.0193573404797334, CR = 0.0215081560885926. Since CI < 0.1, the indicator ranking is reasonable.

[0087] Further calculations showed that the completeness weight, accuracy weight, and consistency weight were 0.6333, 0.1062, and 0.2605, respectively.

[0088] S3: Calculate the overall quality score of the dataset based on the data quality score of each record.

[0089] In this embodiment, the overall quality score of the data set is calculated according to the following formula:

[0090] T=(t1+t2+t3……+tn) / n

[0091] In the formula, T represents the overall quality score of the data set, t1, t2, t3, ... tn represent the data quality scores of each record, and n represents the total number of records in the data set.

[0092] S4: Determine whether the overall quality score is higher than a preset quality threshold. If so, execute S7; if not, execute S5.

[0093] In this embodiment, the data quality qualified threshold is preset as: W=90.

[0094] S5: Filter out unqualified records in the data set based on the overall quality score and the data quality score of each record.

[0095] S51: Calculate the qualified deviation value of each record based on the overall quality score and the data quality score of each record.

[0096] In this embodiment, the qualified deviation value of each record is calculated according to the following formula:

[0097] d=Tt-γ*δ

[0098] Where d represents the qualified deviation value, T represents the overall quality score, t represents the data quality score, γ represents the deviation coefficient, and δ represents the overall standard deviation of the data quality score of each record.

[0099] In this embodiment, the deviation coefficient is set to: γ = 2

[0100] S52: judging whether each record is qualified according to the qualified deviation value of each record; if the qualified deviation value of a certain record is greater than zero, the record is judged as an unqualified record; otherwise, the record is judged as a qualified record.

[0101] S6: Delete or repair unqualified records in the data set to obtain a new data set, and return to S2.

[0102] It should be noted that unqualified records are given priority for repair. The corresponding electricity work order data is queried in other business systems based on the work order number, and then the defects in the unqualified records are supplemented and corrected. If the electricity work order data is an error that already exists in the original system and is not caused by improper collection and cannot be repaired, the unqualified record will be deleted.

[0103] S7: Output the dataset as a governed dataset.

[0104] It is understandable that, in the single record calculation stage, the present invention can perform calculations separately when each record is stored in the library to adapt to the calculation method of real-time calculation, or can perform unified calculations on all records when the data is stored as a whole to adapt to the calculation method of batch calculation; in the overall data calculation stage, due to the small amount of calculation involved, for data sets that are not very large in scale, even its completeness, accuracy, consistency and timeliness can be calculated in quasi-real time. And because the integrity, accuracy, consistency and timeliness of each record are independently marked, when part of the data is updated, only the result of the data can be recalculated, which greatly reduces the scale of computing resources used in related calculations, and the independent marking also helps to locate the position of abnormal records, and is also helpful for subsequent abnormal data repair. Even if the abnormal data is not repaired, the overall data quality of the data set can be improved by excluding low-scoring data records to meet the calculation requirements. This method can not only meet the requirements of evaluation quality, but also greatly save the computing power resources occupied during evaluation.

[0105] The following examples are given to illustrate the method described in this embodiment:

[0106] The work order from the complaint platform has the following data after the field format is adjusted, redundant fields are trimmed, and missing fields are spliced:

[0107]

[0108] The work order from the emergency repair platform has the following data after the field format is adjusted, redundant fields are trimmed, and missing fields are spliced:

[0109]

[0110] (1) The data set obtained by merging the above-mentioned power work order data according to S1 and removing duplicates according to the work order number is as follows:

[0111]

[0112] It should be noted that since there are two records with work order number 10008999, the record with the latest data time is selected.

[0113] (2) Calculate the data quality score of each record in the data set based on S21. The results are as follows:

[0114] For work order 10008997, the content of the request is empty, which is a fatal defect in integrity. The format of the work order initiation time is incorrect, which is a general defect in accuracy. The integrity score of this record = 100-0*3-0*9-0*27-1*81 = 19, the accuracy score = 100-0*3-1*9-0*27-0*81 = 91, and the consistency score = 100-0*3-0*9-0*27-0*81 = 100.

[0115] Work order 10008998, the power consumption agreement capacity is not a value greater than 0 and less than or equal to 999999, which is a general defect of accuracy. The user's power consumption area, Xinglong Street, is not managed by the user's management agency, Nanjing Jiangbei New District Power Supply Branch, which is a serious defect of consistency. The integrity score of this record = 100-0*3-0*9-0*27-0*81 = 100, the accuracy score = 100-0*3-1*9-0*27-0*81 = 91, and the consistency score = 100-0*3-0*9-1*27-0*81 = 73.

[0116] For work order 10008999, the user name is empty, which is a minor defect in completeness. The work order initiation time is later than the work order completion time, which is a serious defect in consistency. The completeness score of this record = 100-1*3-0*9-0*27-0*81 = 97, the accuracy score = 100-0*3-0*9-0*27-0*81 = 100, and the consistency score = 100-0*3-0*9-1*27-0*81 = 73.

[0117] The data quality score of each record is calculated according to S22, and the results are as follows:

[0118] Work order 10008997, data quality score = 19*0.6333+91*0.1062+100*0.2605≈48.

[0119] Work order 10008998, data quality score = 100*0.6333+91*0.1062+73*0.2605≈92.

[0120] Work order 10008999, data quality score = 97*0.6333+100*0.1062+73*0.2605≈91.

[0121] (3) Calculate the overall quality score of the dataset based on S3. The results are as follows:

[0122] Overall quality score = (48 + 92 + 91) / 3 = 77.

[0123] (4) According to S4, determine whether the overall quality score is higher than the qualified quality threshold. The results are as follows:

[0124] 77<90, that is, the overall quality score is less than the preset quality qualification threshold, so S5 is executed.

[0125] (5) According to S5, unqualified records in the data set are screened out, and the results are as follows:

[0126] Work order 10008997, qualified deviation value = 77-48-1.2*20 = 5, greater than zero, unqualified.

[0127] Work order 10008997, qualified deviation value = 77-92-1.2*20 = -39, not greater than zero, qualified.

[0128] Work order 10008997, qualified deviation value = 77-91-1.2*20 = -38, not greater than zero, qualified.

[0129] (6) According to S6, delete or repair the unqualified records in the data set to obtain a new data set. The result is as follows:

[0130] Delete the data of work order 10008997. The data set is as follows:

[0131]

[0132]

[0133] After this step is completed, the process returns to S2, and the calculation process of S2 and S3 is omitted. In S4, 92>90, that is, the overall quality score is greater than the preset quality qualification threshold, so S7 is executed.

[0134] (7) According to S7, the data set is output as a managed data set. The result is as follows:

[0135] Output the current dataset (the new dataset obtained after deleting the data of work order 10008997) as the managed dataset.

[0136] Embodiment 2:

[0137] This embodiment provides a power work order data quality management system, including:

[0138] A data set acquisition module, used to acquire and merge power work order data from different data sources to obtain a data set;

[0139] The data quality calculation module is used to calculate the data quality score of each record in the data set according to the predefined field defect judgment rules and defect level definitions;

[0140] The overall quality calculation module is used to calculate the overall quality score of the data set based on the data quality score of each record;

[0141] The overall quality judgment module is used to judge whether the overall quality score is higher than the preset quality qualification threshold. If so, the data set output module is executed; if not, the data record screening module is executed;

[0142] The data record screening module is used to screen out unqualified records in the data set based on the overall quality score and the data quality score of each record;

[0143] The data record revision module is used to delete or repair unqualified records in the data set to obtain a new data set and return it to the overall quality calculation module;

[0144] The dataset output module is used to output the dataset as a managed dataset.

[0145] Embodiment three:

[0146] This embodiment provides a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by a computing device, the computing device executes any one of the methods described in Embodiment 1.

[0147] Embodiment 4:

[0148] This embodiment provides a computing device, including:

[0149] One or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the methods described in Example 1.

[0150] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0151] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0154] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for managing the quality of power work order data, characterized in that: The following steps are involved: S1: Obtain and merge power work order data from different data sources to obtain a data set; S2: Calculate the data quality score of each record in the data set according to the predefined field defect judgment rules and defect level definition; The data quality score of each record in the calculation data set includes: S21: Calculate the index score of each record according to the field defect judgment rule and the defect level definition, wherein the index score includes a completeness score, an accuracy score, and a consistency score; The calculation of the index score of each record includes: Find the defects that affect the indicator score in each record according to the field defect judgment rules; According to the defect level definition, the defect is classified into defect levels, wherein the defect levels include minor, general, severe and fatal; The indicator score of each record is calculated according to the following formula: ； In the formula, Indicates the indicator score, Indicates the number of minor defects, represents the number of general defects, Indicates the number of serious defects, represents the number of fatal defects, Indicates the deduction value for minor defects. Indicates the deduction value for general defects. Indicates the deduction value for serious defects, Indicates the penalty value for fatal defects; S22: Calculate the data quality score of each record according to the indicator score of each record; S3: Calculate the overall quality score of the data set based on the data quality score of each record; S4: Determine whether the overall quality score is higher than a preset quality threshold, if yes, execute S7, if no, execute S5; S5: Filter out unqualified records in the data set based on the overall quality score and the data quality score of each record; The unqualified records in the data set are screened out as follows: S51. Calculate the qualified deviation value of each record according to the overall quality score and the data quality score of each record; The calculation of the qualified deviation value of each record includes: The qualified deviation value of each record is calculated according to the following formula: ； In the formula, Indicates the qualified deviation value, Represents the overall quality score, represents the data quality score, represents the coefficient of variation, Represents the overall standard deviation of the data quality scores of each record; S52, judging whether each record is qualified according to the qualified deviation value of each record, if the qualified deviation value of a certain record is greater than zero, then the record is judged as an unqualified record, otherwise, the record is judged as a qualified record; S6: Delete or repair unqualified records in the data set to obtain a new data set, and return to S2; S7: Output the dataset as a governed dataset.

2. The method for managing the quality of electric power work order data according to claim 1, characterized in that: Calculating the data quality score of each record according to the indicator score of each record includes: The data quality score of each record is calculated according to the following formula: ； In the formula, represents the data quality score, represents the completeness score, represents the accuracy score, represents the consistency score, represents the integrity weight, represents the accuracy weight, Represents the consistency weight.

3. The method for managing the quality of electric power work order data according to claim 1, characterized in that: Calculating the overall quality score of the data set based on the data quality score of each record includes: The overall quality score of the dataset is calculated according to the following formula: ； In the formula, represents the overall quality score of the dataset, Indicates the data quality score of each record. Indicates the total number of records in the dataset.

4. A power work order data quality management system, characterized in that: include: A data set acquisition module, used to acquire and merge power work order data from different data sources to obtain a data set; The data quality calculation module is used to calculate the data quality score of each record in the data set according to the predefined field defect judgment rules and defect level definitions; The data quality score of each record in the calculation data set includes: S21: Calculate the index score of each record according to the field defect judgment rule and the defect level definition, wherein the index score includes a completeness score, an accuracy score, and a consistency score; The calculation of the index score of each record includes: Find the defects that affect the indicator score in each record according to the field defect judgment rules; According to the defect level definition, the defect is classified into defect levels, wherein the defect levels include minor, general, severe and fatal; The indicator score of each record is calculated according to the following formula: ； In the formula, Indicates the indicator score, Indicates the number of minor defects, represents the number of general defects, Indicates the number of serious defects, represents the number of fatal defects, Indicates the deduction value for minor defects. Indicates the deduction value for general defects. Indicates the deduction value for serious defects, Indicates the penalty value for fatal defects; S22: Calculate the data quality score of each record according to the indicator score of each record; The overall quality calculation module is used to calculate the overall quality score of the data set based on the data quality score of each record; The overall quality judgment module is used to judge whether the overall quality score is higher than the preset quality qualification threshold. If so, the data set output module is executed; if not, the data record screening module is executed; The data record screening module is used to screen out unqualified records in the data set based on the overall quality score and the data quality score of each record; The unqualified records in the data set are screened out as follows: S51. Calculate the qualified deviation value of each record according to the overall quality score and the data quality score of each record; The calculation of the qualified deviation value of each record includes: The qualified deviation value of each record is calculated according to the following formula: ； In the formula, Indicates the qualified deviation value, Represents the overall quality score, represents the data quality score, represents the coefficient of variation, Represents the overall standard deviation of the data quality scores of each record; S52, judging whether each record is qualified according to the qualified deviation value of each record, if the qualified deviation value of a certain record is greater than zero, then the record is judged as an unqualified record, otherwise, the record is judged as a qualified record; The data record revision module is used to delete or repair unqualified records in the data set to obtain a new data set and return it to the overall quality calculation module; The dataset output module is used to output the dataset as a managed dataset.

5. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions which, when executed by a computing device, cause the computing device to perform any one of the methods according to claims 1 to 3.

6. A computing device, characterized in that include: One or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the methods according to claims 1 to 3.

Images