Provident Fund Data Quality Assessment and Grading Early Warning System and Method

By cleaning and quality-assessing the housing provident fund data, and using hash functions and deep neural network models to identify and verify the data, early warning instructions are generated, solving the problem of missing rules in the assessment of housing provident fund data and achieving efficient data management and business decision support.

CN118608077BActive Publication Date: 2024-11-19JIANGSU FINSTONE TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410737842.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-07
Publication Date
2024-11-19
Estimated Expiration
2044-06-07

AI Technical Summary

Technical Problem

Existing methods for evaluating housing provident fund data fail to provide specific evaluation rules and standards, and cannot effectively handle the problems of duplication, errors and inconsistencies in large-scale, multi-source housing provident fund data, thus affecting the scientific nature and efficiency of business decisions.

Method used

By cleaning and quality assessment of the housing provident fund system data, using hash functions to identify duplicate and unreasonable data, combining deep neural network models to judge the rationality of date and transaction data, and verifying the accuracy of data through multiple query servers, different levels of early warning instructions are generated.

Benefits of technology

It has enabled automated cleaning and quality management of housing provident fund data, improved the overall data quality level, reduced the burden of manual review, and enhanced the scientific nature and accuracy of business decisions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118608077B_ABST
    Figure CN118608077B_ABST
Patent Text Reader

Abstract

The present invention belongs to the technical field of data quality assessment, and discloses a provident fund data quality assessment and graded early warning system and method; the system comprises: collecting m groups of assessment data from a provident fund system, where m is an integer greater than 1; performing data cleaning on the m groups of assessment data; performing quality assessment on the m groups of assessment data after data cleaning, and automatically identifying quality problems; generating corresponding early warning instructions for the assessment data that has undergone data cleaning and has identified quality problems; the present invention comprehensively checks problem data through a variety of data verification means, realizes automation and efficient processing of provident fund data quality management work, greatly improves the overall quality level of provident fund data, reduces the burden of manual review, and thus improves the scientificity and accuracy of business decisions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data quality assessment, and more specifically, to a system and method for providing housing fund data quality assessment and graded early warning. Background Art

[0002] In the provident fund management system, provident fund data is an important reference for economic activities. The quality of provident fund data directly affects the scientificity and accuracy of business decisions, as well as the efficiency of business processes. With the improvement of my country's social and economic activity, the number of public servants continues to grow. Provident fund centers in various regions need to process a huge amount of provident fund personal account data every month. These data come from multiple departments and units. Due to the large amount of data and complex sources, data duplication, errors and inconsistencies may occur.

[0003] In the prior art, there is an intelligent provident fund data evaluation method. For example, the Chinese patent with publication number CN117252685A discloses a provident fund information review method, system, device, equipment and storage medium; including: controlling the RPA executor at the data collection end to collect data on the provident fund information of the customer to be reviewed, and obtaining the collected data of the customer to be reviewed; using the image text recognition technology to perform image text recognition on the image to be recognized in the collected data, and obtaining the text recognition result of the customer to be reviewed; based on the text recognition result, the provident fund information of the customer to be reviewed is reviewed; based on the RPA technology to simulate human actions, the provident fund information of the customer is collected, and the provident fund information of the bank customer is reviewed based on the collected data, which effectively avoids the risk of human error, saves manpower and material resources, speeds up business processing speed and response capabilities, and also improves the accuracy and efficiency of the provident fund information review;

[0004] However, the above technology focuses on how to obtain the collected data of customers to be reviewed, and does not specifically introduce the relevant process of provident fund information review, and it is impossible to obtain specific provident fund data evaluation rules and standards;

[0005] In view of this, the present invention proposes a provident fund data quality assessment and graded warning system and method to solve the above problems. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for providing housing fund data quality assessment and graded early warning, comprising:

[0007] Collect m groups of assessment data from the provident fund system, where m is an integer greater than 1;

[0008] Perform data cleaning on m groups of evaluation data;

[0009] Perform quality assessment on the m groups of evaluation data after data cleaning and automatically identify quality issues;

[0010] Generate corresponding early warning instructions for data cleaning and evaluation data with quality issues identified.

[0011] Furthermore, the evaluation data includes at least personal information, transaction records and balance records;

[0012] The personal information includes the name, ID number, telephone number and address of the provident fund account holder;

[0013] The transaction records include deposit records and withdrawal records;

[0014] The deposit record includes the single deposit amount, single deposit date and deposit unit;

[0015] The withdrawal record includes a single withdrawal amount and a single withdrawal date;

[0016] The balance record is the balance information in the provident fund account;

[0017] A set of assessment data corresponds to a provident fund account in the provident fund system.

[0018] Furthermore, the step of performing data cleaning on the m groups of evaluation data includes:

[0019] Step 1: Delete duplicate data in m groups of evaluation data;

[0020] Step 2: Delete unreasonable data from the m groups of evaluation data.

[0021] Furthermore, the method for deleting duplicate data in m groups of evaluation data includes:

[0022] For each data in each group of personal information in the m groups of evaluation data, a corresponding hash value is calculated using a hash function and marked as a personal hash value; each personal hash value is compared to determine whether there is an identical personal hash value; if there is an identical personal hash value, the data with the same personal hash value is marked as duplicate data, one duplicate data among the n duplicate data with the same personal hash value is retained, and the remaining n-1 duplicate data are deleted from the m groups of evaluation data, where n is an integer greater than 1;

[0023] If the name and ID number in the personal information in one set of evaluation data are deleted, the corresponding evaluation data will be deleted from the m sets of evaluation data.

[0024] Furthermore, the step of deleting unreasonable data in the m groups of evaluation data includes:

[0025] Step 201: Determine whether to mark the ID card number or telephone number in each set of personal information in the m sets of evaluation data as unreasonable data;

[0026] For the ID card number and telephone number in each group of personal information in the m groups of evaluation data, a string function is used to obtain the corresponding string length, and the string length corresponding to the ID card number is marked as the ID card length, and the string length corresponding to the telephone number is marked as the telephone length; the ID card length threshold and the telephone length threshold are preset, and each ID card length is compared with the ID card length threshold. If the ID card length is inconsistent with the ID card length threshold, the corresponding ID card number is marked as unreasonable data; if the ID card length is consistent with the ID card length threshold, the corresponding ID card number is not marked as unreasonable data; each telephone length is compared with the telephone length threshold. If the telephone length is inconsistent with the telephone length threshold, the corresponding telephone number is marked as unreasonable data; if the telephone length is consistent with the telephone length threshold, the corresponding telephone number is not marked as unreasonable data;

[0027] Step 202: Determine whether to mark the single deposit amount, single withdrawal amount or balance record in the m groups of evaluation data as unreasonable data;

[0028] The single deposit amount, single withdrawal amount and balance record in the m group of evaluation data are analyzed; if the single deposit amount is less than 0, the corresponding single deposit amount is marked as unreasonable data; if the single deposit amount is greater than or equal to 0, the corresponding single deposit amount is not marked as unreasonable data; if the single withdrawal amount is less than 0, the corresponding single withdrawal amount is marked as unreasonable data; if the single withdrawal amount is greater than or equal to 0, the corresponding single withdrawal amount is not marked as unreasonable data; if the balance record is less than 0, the corresponding balance record is marked as unreasonable data; if the balance record is greater than or equal to 0, the corresponding balance record is not marked as unreasonable data;

[0029] Step 203: Determine whether to mark a single deposit date or a single withdrawal date in the m groups of evaluation data as unreasonable data;

[0030] Mark the single deposit date and single withdrawal date in the m group of evaluation data as date data, input the date data into the trained date analysis model, and determine whether the date data is unreasonable;

[0031] The training process of the date analysis model includes:

[0032] Pre-set corresponding judgment results for a date data, where a is an integer greater than 1, and the judgment results include reasonable dates and unreasonable dates, and different digital labels are set for reasonable dates and unreasonable dates;

[0033] The digital labels of the judgment results are marked as judgment labels, and the date data and the corresponding judgment labels are converted into a corresponding set of feature vectors;

[0034] Each group of feature vectors is used as the input of the date analysis model. The date analysis model uses a group of prediction judgment labels corresponding to each group of date data as output, and uses the actual judgment labels corresponding to each group of date data as the prediction target, and the actual judgment labels are the digital labels of the judgment results corresponding to the date data pre-set above; minimizing the sum of the prediction errors of all date data is used as the training target; the date analysis model is trained until the sum of the prediction errors reaches convergence and the training is stopped; the date analysis model is a deep neural network model;

[0035] Obtain the corresponding judgment result according to the predicted judgment label; if the judgment result is that the date is reasonable, the corresponding date data is not marked as unreasonable data; if the judgment result is that the date is unreasonable, the corresponding date data is marked as unreasonable data;

[0036] Step 204: Delete unreasonable data from the m groups of evaluation data.

[0037] Furthermore, the method for performing quality assessment on the m groups of evaluation data after data cleaning includes:

[0038] The ID card number in the m group of evaluation data is sent to the query server, which includes a social security query server, a household registration query server and an operator query server; wherein the social security query server queries according to the ID card number, generates the corresponding name and deposit unit, and gives feedback; the household registration query server queries according to the ID card number, generates the corresponding address, and gives feedback; the operator query server queries according to the ID card number, generates the corresponding telephone number, and gives feedback;

[0039] For the name, deposit unit, address and telephone number obtained from the feedback, the corresponding hash values ​​are calculated using a hash function and marked as comparison hash values; the corresponding hash values ​​of the deposit units in the m groups of evaluation data are then calculated using a hash function and also marked as personal hash values;

[0040] Construct an analysis set from the comparison hash value corresponding to an ID card number; construct an evaluation set from the personal hash value corresponding to an ID card number; mark the name, deposit unit, address and telephone number as analysis data; compare the analysis set and evaluation set with the same corresponding ID card number, and compare each comparison hash value in the analysis set with the corresponding personal hash value in the evaluation set;

[0041] If the comparison hash value is the same as the corresponding personal hash value, the analysis data corresponding to the comparison hash value and the personal hash value is not marked as problematic data;

[0042] If the comparison hash value is different from the corresponding personal hash value, the analysis data corresponding to the comparison hash value and the personal hash value are marked as problematic data;

[0043] If the analysis data corresponding to each comparison hash value in an analysis set are all marked as problem data, then the corresponding analysis set is marked as a problem set, and the ID number corresponding to the problem set is marked as problem data; then the phone number corresponding to the problem set is sent to the operator query server, and the operator query server queries based on the phone number, generates the corresponding name and gives feedback; the hash value corresponding to the fed-back name is calculated using a hash function and marked as a secondary comparison hash value; the name corresponding to the problem set is marked as a comparison name, and the comparison hash value corresponding to the comparison name is compared with the secondary comparison hash value; if the comparison hash value is consistent with the secondary comparison hash value, the phone number and name corresponding to the problem set are not marked as problem data; if the comparison hash value is inconsistent with the secondary comparison hash value, the phone number and name corresponding to the problem set are still marked as problem data;

[0044] If the ID number in the assessment data is deleted, the name, deposit unit, address and telephone number in the corresponding assessment data that have not been deleted will be marked as problematic data.

[0045] Furthermore, the warning instructions include high-level warning instructions and low-level warning instructions; the high-level warning instructions include first-level high-level warning instructions and second-level high-level warning instructions.

[0046] Furthermore, the method for generating corresponding warning instructions for the evaluation data for data cleaning includes:

[0047] If an ID number or name in a set of evaluation data is marked as problematic data or deleted, a critical error instruction is generated;

[0048] If both the ID number and the name in a set of evaluation data are marked as problematic data or deleted at the same time, a multi-key error instruction is generated;

[0049] If a transaction record, phone number or address in a set of evaluation data is marked as problematic data or deleted, a general error instruction is generated;

[0050] If transaction records, phone numbers or addresses in a set of evaluation data are all marked as problematic data or deleted, multiple general error instructions are generated;

[0051] If multiple critical error instructions are generated in a set of evaluation data, a second-level advanced warning instruction is generated;

[0052] If multiple critical error instructions are not generated in a set of evaluation data, but critical error instructions are generated, a first-level advanced warning instruction is generated;

[0053] If multiple critical error instructions and key error instructions are not generated in a set of evaluation data, but multiple general error instructions are generated, a first-level advanced warning instruction is generated;

[0054] If multiple critical error instructions, critical error instructions and multiple general error instructions are not generated in a set of evaluation data, but general error instructions are generated, a low-level warning instruction is generated.

[0055] Furthermore, the evaluation data corresponding to the names with the same personal hash values ​​are analyzed to determine whether they are marked as duplicate data;

[0056] Mark the evaluation data corresponding to names with the same personal hash value as identical data;

[0057] If the ID numbers in the same data in group b are all marked as duplicate data, then the names in the same data in group b are also marked as duplicate data, where b is an integer greater than 1;

[0058] If there are ID card numbers that are not marked as duplicate data in the b groups of identical data, the personal hash values ​​corresponding to each group of identical data are used as a set of test sets, that is, the test sets correspond to the identical data one by one; and the identical data corresponding to the ID card numbers that are not marked as duplicate data are marked as different data; digital labels are set incrementally for the test sets corresponding to the identical data and the test sets corresponding to the different data, and the digital labels corresponding to the test sets are marked as test labels, and the range of the test labels is [1, b′], b′=b;

[0059] The test labels corresponding to the same data and the test labels corresponding to a group of different data are taken as a group of total test sets; each group of total test sets is input into the trained probability prediction model in turn to predict the corresponding repetition probability;

[0060] The training process of the probabilistic prediction model includes:

[0061] Pre-collect the repetition probabilities corresponding to the total test set of group b, and convert the total test set and the corresponding repetition probabilities into a corresponding set of feature vectors;

[0062] Each group of feature vectors is used as the input of the probability analysis model. The probability analysis model uses a group of repetition probabilities corresponding to each group of total test sets as output, and uses the actual repetition probability corresponding to each group of total test sets as the prediction target, where the actual repetition probability is the pre-collected repetition probability corresponding to the total test set; minimizing the sum of prediction errors of all total test sets is used as the training target; the probability analysis model is trained until the sum of prediction errors reaches convergence and the training is stopped; the probability analysis model is a deep neural network model;

[0063] Preset probability threshold P T ;

[0064] The repetition probability P C With probability threshold P T Make a comparison;

[0065] If P C ≤P T , then the names in different data are not marked as duplicate data, and the names in the same data are marked as duplicate data;

[0066] If P C >P T , then the names in different data and the names in the same data are marked as duplicate data.

[0067] The provident fund data quality assessment and graded early warning system implements the provident fund data quality assessment and graded early warning method, including:

[0068] A data collection module is used to collect m groups of evaluation data from the provident fund system, where m is an integer greater than 1;

[0069] A data preprocessing module is used to clean the m groups of evaluation data;

[0070] The data quality assessment module is used to perform quality assessment on the m groups of evaluation data after data cleaning and automatically identify quality problems;

[0071] The early warning module is used to generate corresponding early warning instructions for data cleaning and evaluation data with quality problems identified.

[0072] The technical effects and advantages of the provident fund data quality assessment and graded early warning system and method of the present invention are as follows:

[0073] 1. It can clean and evaluate the quality of provident fund data, automatically generate early warning instructions of different levels according to quality issues, and send them to relevant personnel electronically; comprehensively check the problematic data through various data verification methods, realize the automation and efficient processing of provident fund data quality management, greatly improve the overall quality level of provident fund data, reduce the burden of manual review, and thus improve the scientificity and accuracy of business decisions.

[0074] 2. By analyzing the evaluation data corresponding to names with the same personal hash values, using the probability prediction model to predict the probability of duplication, and conducting quantitative analysis on the judgment results, it is possible to more accurately determine whether to mark the name as duplicate data; it can better coordinate the relationship between duplicate data identification and normal data retention, so that data cleaning work can take into account both duplicate data removal and information loss minimization, effectively improving the efficiency and level of provident fund data cleaning, thereby improving the accuracy of provident fund data evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 This is a schematic diagram of the structure of the provident fund data quality assessment and graded early warning system according to Example 1 of the present invention;

[0076] Figure 2 This is a schematic diagram of the structure of a provident fund data quality assessment and graded early warning system according to Embodiment 2 of the present invention;

[0077] Figure 3 This is a flow chart of a method for providing housing fund data quality assessment and graded early warning according to Embodiment 3 of the present invention;

[0078] Figure 4 This is a schematic diagram of an electronic device according to Embodiment 4 of the present invention;

[0079] Figure 5 This is a schematic diagram of a storage medium according to Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0081] Example 1

[0082] See also Figure 1 As shown, the provident fund data quality assessment and graded warning system described in this embodiment includes a data collection module, a data preprocessing module, a data quality assessment module and a warning module; each module is connected by wired and / or wireless means to achieve data transmission between modules.

[0083] The data collection module is used to collect m groups of evaluation data from the provident fund system, where m is an integer greater than 1.

[0084] The evaluation data includes at least personal information, transaction records and balance records.

[0085] Personal information includes the CPF account holder’s name, ID number, telephone number and address.

[0086] Transaction records include deposit records and withdrawal records.

[0087] The deposit record includes the single deposit amount, single deposit date and deposit unit.

[0088] Withdrawal records include single withdrawal amount and single withdrawal date.

[0089] The balance record is the balance information in the provident fund account.

[0090] A set of assessment data corresponds to a provident fund account in the provident fund system.

[0091] The data preprocessing module is used to clean the m groups of evaluation data.

[0092] The steps for data cleaning of the m group evaluation data include:

[0093] Step 1: Delete duplicate data from the m groups of evaluation data.

[0094] Methods for deleting duplicate data in m groups of evaluation data include:

[0095] For each data in each group of personal information in the m groups of evaluation data, a hash function is used to calculate the corresponding hash value and marked as a personal hash value. The hash function may be SM2, SM3, SHA3-512, Whirlpool, etc. Each personal hash value is compared to determine whether there is an identical personal hash value. If there is an identical personal hash value, the data with the same personal hash value is marked as duplicate data, and one of the n duplicate data with the same personal hash value is retained, and the remaining n-1 duplicate data are deleted from the m groups of evaluation data, where n is an integer greater than 1.

[0096] If the name and ID number in the personal information in one set of evaluation data are deleted, the corresponding evaluation data will be deleted from the m sets of evaluation data; the reason is that if the name and ID number in the personal information in multiple sets of evaluation data are the same, it means that the provident fund system has stored duplicate data for a provident fund account, and the redundant data needs to be deleted.

[0097] Step 2: Delete unreasonable data from the m groups of evaluation data.

[0098] The steps of deleting unreasonable data from the m groups of evaluation data include:

[0099] Step 201: Determine whether to mark the ID card number or telephone number in each set of personal information in the m sets of evaluation data as unreasonable data.

[0100] For the ID card number and telephone number in each group of personal information in the m groups of evaluation data, a string function is used to obtain the corresponding string length, and the string length corresponding to the ID card number is marked as the ID card length, and the string length corresponding to the telephone number is marked as the telephone length. String functions such as length and strlen are preset. The ID card length threshold and the telephone length threshold are compared with each ID card length and the ID card length threshold. If the ID card length is inconsistent with the ID card length threshold, the corresponding ID card number is marked as unreasonable data; if the ID card length is consistent with the ID card length threshold, the corresponding ID card number is not marked as unreasonable data; each telephone length is compared with the telephone length threshold. If the telephone length and the telephone length threshold are inconsistent, the corresponding telephone number is marked as unreasonable data; if the telephone length and the telephone length threshold are consistent, the corresponding telephone number is not marked as unreasonable data; the ID card number length threshold and the telephone length threshold are preset by technical personnel in this field according to actual conditions. In this embodiment, the preferred ID card number length threshold is 18 and the telephone length threshold is 11.

[0101] Step 202: Determine whether to mark the single deposit amount, single withdrawal amount or balance record in the m groups of evaluation data as unreasonable data.

[0102] The single deposit amount, single withdrawal amount and balance records in the m groups of evaluation data are all analyzed; if the single deposit amount is less than 0, the corresponding single deposit amount will be marked as unreasonable data; if the single deposit amount is greater than or equal to 0, the corresponding single deposit amount will not be marked as unreasonable data; if the single withdrawal amount is less than 0, the corresponding single withdrawal amount will be marked as unreasonable data; if the single withdrawal amount is greater than or equal to 0, the corresponding single withdrawal amount will not be marked as unreasonable data; if the balance record is less than 0, the corresponding balance record will be marked as unreasonable data; if the balance record is greater than or equal to 0, the corresponding balance record will not be marked as unreasonable data.

[0103] It should be noted that since the single deposit amount, single withdrawal amount and balance record do not exist in the case of less than 0, if there is a single deposit amount, single withdrawal amount or balance record less than 0 in the m groups of evaluation data, it is unreasonable data.

[0104] Step 203: Determine whether to mark a single deposit date or a single withdrawal date in the m groups of evaluation data as unreasonable data.

[0105] Mark the single deposit date and single withdrawal date in the m groups of evaluation data as date data, input the date data into the trained date analysis model to determine whether the date data is unreasonable.

[0106] The specific training process of the date analysis model includes:

[0107] Corresponding judgment results are set for a date data in advance, where a is an integer greater than 1, and the judgment results include a reasonable date and an unreasonable date. Different digital labels are set for the reasonable date and the unreasonable date. Exemplarily, the digital label is set to 0 for the reasonable date, and the digital label is set to 1 for the unreasonable date; the judgment results corresponding to the date data are collected by technical personnel in this field during the historical provident fund data quality assessment process, and a different date data are collected. The technical personnel in this field judge whether a different date data are unreasonable (for example, the date data is January 32, February 30, etc.) in turn according to actual experience, and set corresponding judgment results for a groups of different date data in turn.

[0108] The numerical labels of the judgment results are marked as judgment labels, and the date data and the corresponding judgment labels are converted into a corresponding set of feature vectors.

[0109] Each set of feature vectors is used as the input of the date analysis model. The date analysis model uses a set of prediction judgment labels corresponding to each set of date data as output, and uses the actual judgment labels corresponding to each set of date data as the prediction target. The actual judgment label is the digital label of the judgment result corresponding to the date data set in advance. The training goal is to minimize the sum of the prediction errors of all date data. The calculation formula of the prediction error is Zp = (αp-μp) 2 , where Z p is the prediction error, p is the group number of the feature vector corresponding to the date data, α p is the prediction label corresponding to the p-th group of date data, μ p is the actual judgment label corresponding to the pth group of date data; the date analysis model is trained until the sum of the prediction errors reaches convergence and the training is stopped.

[0110] The above date analysis model is specifically a deep neural network model.

[0111] The corresponding judgment result is obtained according to the predicted judgment label; if the judgment result is that the date is reasonable, the corresponding date data is not marked as unreasonable data; if the judgment result is that the date is unreasonable, the corresponding date data is marked as unreasonable data.

[0112] Step 204: Delete unreasonable data from the m groups of evaluation data.

[0113] The data quality assessment module is used to perform quality assessment on the m groups of evaluation data after data cleaning and automatically identify quality problems.

[0114] The method for performing quality assessment on the m groups of evaluation data after data cleaning includes:

[0115] The ID card number in the m groups of evaluation data is sent to the query server, which includes a social security query server, a household registration query server and an operator query server, such as a China Unicom query server, a China Telecom query server and a China Mobile query server; among which, the social security query server queries according to the ID card number, generates the corresponding name and deposit unit, and provides feedback; the household registration query server queries according to the ID card number, generates the corresponding address, and provides feedback; the operator query server queries according to the ID card number, generates the corresponding telephone number, and provides feedback.

[0116] For the name, deposit unit, address and telephone number obtained from the feedback, the corresponding hash values ​​are calculated using a hash function and marked as comparison hash values; then the corresponding hash value is calculated for the deposit unit in the m groups of evaluation data using a hash function and also marked as a personal hash value.

[0117] An analysis set is constructed from the comparison hash value corresponding to an ID card number; an evaluation set is constructed from the personal hash value corresponding to an ID card number; the name, deposit unit, address and telephone number are marked as analysis data; the analysis set and the evaluation set with the same corresponding ID card number are compared, and each comparison hash value in the analysis set is compared with the corresponding personal hash value in the evaluation set, that is, the analysis data corresponding to the compared comparison hash value and the personal hash value are the same, for example, both correspond to the name, deposit unit, etc.

[0118] If the comparison hash value is the same as the corresponding personal hash value, the analysis data corresponding to the comparison hash value and the personal hash value will not be marked as problematic data, indicating that there is no quality problem with the corresponding data.

[0119] If the comparison hash value is different from the corresponding personal hash value, the analysis data corresponding to the comparison hash value and the personal hash value will be marked as problematic data, indicating that there are quality problems with the corresponding analysis data, and that the analysis data stored in the provident fund system is incorrect and needs to be changed.

[0120] If the analysis data corresponding to each comparison hash value in an analysis set are all marked as problem data, then the corresponding analysis set will be marked as a problem set, and the ID number corresponding to the problem set will be marked as problem data; the phone number corresponding to the problem set will then be sent to the operator query server, and the operator query server will query based on the phone number, generate the corresponding name and provide feedback; the corresponding hash value of the fed-back name will be calculated using a hash function and marked as a secondary comparison hash value; the name corresponding to the problem set will be marked as a comparison name, and the comparison hash value corresponding to the comparison name will be compared with the secondary comparison hash value; if the comparison hash value is consistent with the secondary comparison hash value, the phone number and name corresponding to the problem set will not be marked as problem data; if the comparison hash value is inconsistent with the secondary comparison hash value, the phone number and name corresponding to the problem set will still be marked as problem data.

[0121] It should be noted that the purpose of querying names based on phone numbers is that, although the analysis data corresponding to the problem set are all marked as problem data, this may be caused by an incorrect ID number. Therefore, by querying names based on phone numbers, it is possible to determine whether the phone number corresponds to the name. If the phone number and name correspond, it means that the reason the phone number and name are marked as problem data is that the ID number is incorrect. Therefore, there is no need to mark the phone number and name as problem data.

[0122] If the ID number in the assessment data is deleted, the name, deposit unit, address and telephone number in the corresponding assessment data that have not been deleted will be marked as problematic data.

[0123] The early warning module is used to generate corresponding early warning instructions for data cleaning and evaluation data with quality problems identified.

[0124] Early warning instructions include high-level early warning instructions and low-level early warning instructions; among them, high-level early warning instructions include first-level high-level early warning instructions and second-level high-level early warning instructions.

[0125] If an ID card number or name in a set of evaluation data is marked as problematic data or deleted, a key error instruction is generated, indicating that an error has occurred in key data in the evaluation data.

[0126] If the ID card number and name in a set of evaluation data are marked as problem data or deleted at the same time, a multi-key error instruction is generated; this indicates that errors occur in all key data in the evaluation data.

[0127] If a transaction record, telephone number or address in a set of evaluation data is marked as problematic data or deleted, a general error instruction is generated, indicating that an error has occurred in non-critical data in the evaluation data.

[0128] If all transaction records, phone numbers or addresses in a set of evaluation data are marked as problematic data or deleted, then multiple general error instructions are generated, indicating that all non-critical data in the evaluation data are erroneous;

[0129] If multiple key error instructions are generated in a set of evaluation data, a second-level advanced warning instruction is generated, indicating that all key data in the set of evaluation data have errors and modification has the highest priority.

[0130] If multiple critical error instructions are not generated in a set of evaluation data, but critical error instructions are generated, a first-level advanced warning instruction is generated, indicating that errors occur in critical data in the set of evaluation data, but not all critical data, and the priority for modification is higher.

[0131] If a group of evaluation data does not generate multiple critical error instructions and critical error instructions, but generates multiple general error instructions, a first-level advanced warning instruction is generated, indicating that all non-critical data in the group of evaluation data have errors, and the priority for modification is also high.

[0132] If a set of evaluation data does not generate multiple critical error instructions, critical error instructions, and multiple general error instructions, but generates general error instructions, a low-level warning instruction is generated, indicating that non-critical data in the set of evaluation data has errors and the priority for modification is low.

[0133] The generated early warning instructions are sent to relevant business personnel via email, text messages, etc., so that the relevant business personnel can modify and process the evaluation data in a timely and effective manner according to the early warning instructions.

[0134] This embodiment can clean and evaluate the quality of provident fund data, automatically generate early warning instructions of different levels according to quality problems, and send them to relevant personnel electronically; comprehensively check the problem data through various data verification methods, realize the automation and efficient processing of provident fund data quality management, greatly improve the overall quality level of provident fund data, reduce the burden of manual review, and thus improve the scientificity and accuracy of business decisions.

[0135] Example 2

[0136] See also Figure 2 As shown, this embodiment further improves the design on the basis of Embodiment 1. Since there may be duplication of names in personal information, that is, the names of multiple people are the same, when deleting duplicate data in Embodiment 1, the names that are repeated but have no quality problems in the m groups of evaluation data will be deleted; therefore, this embodiment provides a provident fund data quality assessment and graded early warning system, which also includes a data analysis module.

[0137] The data analysis module is used to analyze the evaluation data corresponding to the names with the same personal hash value to determine whether to mark them as duplicate data.

[0138] The evaluation data corresponding to names with the same personal hash value are marked as identical data.

[0139] If the ID card numbers in the same data in group b are all marked as duplicate data, then the names in the same data in group b are also marked as duplicate data, and b is an integer greater than 1; this means that the same data in group b all represent a provident fund account.

[0140] If there are identity card numbers that are not marked as duplicate data in the b groups of identical data, the personal hash values ​​corresponding to each group of identical data are used as a set of test sets, that is, the test sets correspond to the identical data one-to-one; and the identical data corresponding to the identity card numbers that are not marked as duplicate data are marked as different data; digital labels are incrementally set for the test sets corresponding to the identical data and the test sets corresponding to the different data, and the digital labels corresponding to the test sets are marked as test labels, and the range of the test labels is [1, b′], b′=b.

[0141] The test labels corresponding to the same data and the test labels corresponding to a group of different data are taken as a total test set; each total test set is input into the trained probability prediction model in turn to predict the corresponding repetition probability.

[0142] The specific training process of the probability prediction model includes:

[0143] The repetition probabilities corresponding to the total test sets of group b are collected in advance, and the total test sets and the corresponding repetition probabilities are converted into a corresponding set of feature vectors.

[0144] Each set of feature vectors is used as the input of the probability analysis model. The probability analysis model takes a set of repetition probabilities corresponding to each set of total test sets as the output, and takes the actual repetition probability corresponding to each set of total test sets as the prediction target. The actual repetition probability is the repetition probability corresponding to the total test set collected in advance; the training target is to minimize the sum of the prediction errors of all total test sets; where the calculation formula of the prediction error is Z k =(α k -μ k ) 2 , where Zk is the prediction error, k is the group number of the feature vector corresponding to the total test set, αk is the repetition probability corresponding to the kth group of the total test set, and μk is the actual repetition probability corresponding to the kth group of the total test set; the probability analysis model is trained until the sum of the prediction errors converges and the training is stopped.

[0145] The above-mentioned probability analysis model is specifically a deep neural network model, which includes an input layer, a hidden layer and an output layer; each hidden layer includes multiple neurons, each neuron is connected to the neurons in the next layer, and the connection contains weights, which determine the importance and influence of data transmission in the neural network; an activation function is applied to each neuron between the hidden layer and the output layer, and the activation function reflects nonlinearity, allowing the network to learn more complex patterns and features.

[0146] It should be noted that the repetition probability corresponding to the total test set is obtained by technical personnel in the process of historical provident fund data quality assessment, who collect b groups of different total test sets and analyze the corresponding repetition probability under the conditions of each group of total test sets in combination with actual experience.

[0147] Preset probability threshold P T , probability threshold P T During the quality assessment of historical provident fund data, a technician in this field collects c different total test sets, where the names in different data in each total test set are consistent with the names in the same data, but there are no quality problems; the c different total test sets are input into the probability analysis model in turn to obtain the corresponding repetition probability, and the average of the c repetition probabilities is used as the probability threshold P T .

[0148] The repetition probability P C With probability threshold P T for comparison.

[0149] If P C ≤P T , the names in different data are not marked as duplicate data, and the names in the same data are marked as duplicate data; this indicates that although the names in different data are consistent with the names in the same data, there are no quality problems with the names in different data.

[0150] If P C >P T , then the names in different data and the names in the same data are marked as duplicate data; this indicates that the names in different data have quality problems and need to be deleted.

[0151] This embodiment analyzes the evaluation data corresponding to the names with the same personal hash values, predicts the probability of duplication using a probability prediction model, and performs quantitative analysis on the judgment results to more accurately determine whether to mark the names as duplicate data. It can better coordinate the relationship between duplicate data identification and normal data retention, so that data cleaning work takes into account both duplicate data removal and information loss minimization, effectively improving the efficiency and level of provident fund data cleaning, thereby improving the accuracy of provident fund data evaluation.

[0152] Example 3

[0153] See also Figure 3 As shown, the parts not described in detail in this embodiment are described in Example 1 and Example 2, and a method for evaluating the quality of provident fund data and providing a graded early warning is provided, and the method includes:

[0154] Collect m groups of assessment data from the provident fund system, where m is an integer greater than 1.

[0155] Perform data cleaning on m groups of evaluation data.

[0156] Perform quality assessment on the m groups of evaluation data after data cleaning and automatically identify quality issues.

[0157] Generate corresponding early warning instructions for data cleaning and evaluation data with quality issues identified.

[0158] Specifically, the evaluation data includes at least personal information, transaction records and balance records.

[0159] The personal information includes the provident fund account holder’s name, ID number, telephone number and address.

[0160] The transaction records include deposit records and withdrawal records.

[0161] The deposit record includes the single deposit amount, single deposit date and deposit unit.

[0162] The withdrawal record includes a single withdrawal amount and a single withdrawal date.

[0163] The balance record is the balance information in the provident fund account.

[0164] A set of assessment data corresponds to a provident fund account in the provident fund system.

[0165] Specifically, the step of performing data cleaning on the m groups of evaluation data includes:

[0166] Step 1: Delete duplicate data from the m groups of evaluation data.

[0167] Step 2: Delete unreasonable data from the m groups of evaluation data.

[0168] Furthermore, the method for deleting duplicate data in m groups of evaluation data includes:

[0169] For each data in each group of personal information in the m groups of evaluation data, the corresponding hash value is calculated using a hash function and marked as a personal hash value; each personal hash value is compared to determine whether there is an identical personal hash value; if there is an identical personal hash value, the data with the same personal hash value is marked as duplicate data, one duplicate data among the n duplicate data with the same personal hash value is retained, and the remaining n-1 duplicate data are deleted from the m groups of evaluation data, where n is an integer greater than 1.

[0170] If the name and ID number in the personal information in one set of evaluation data are deleted, the corresponding evaluation data will be deleted from the m sets of evaluation data.

[0171] Specifically, the step of deleting unreasonable data in the m groups of evaluation data includes:

[0172] Step 201: Determine whether to mark the ID card number or telephone number in each set of personal information in the m sets of evaluation data as unreasonable data.

[0173] For the ID card number and telephone number in each group of personal information in the m groups of evaluation data, the string function is used to obtain the corresponding string length, and the string length corresponding to the ID card number is marked as the ID card length, and the string length corresponding to the telephone number is marked as the telephone length; the ID card length threshold and the telephone length threshold are preset, and each ID card length is compared with the ID card length threshold. If the ID card length is inconsistent with the ID card length threshold, the corresponding ID card number is marked as unreasonable data; if the ID card length and the ID card length threshold are consistent, the corresponding ID card number is not marked as unreasonable data; each telephone length is compared with the telephone length threshold. If the telephone length and the telephone length threshold are inconsistent, the corresponding telephone number is marked as unreasonable data; if the telephone length and the telephone length threshold are consistent, the corresponding telephone number is not marked as unreasonable data.

[0174] Step 202: Determine whether to mark the single deposit amount, single withdrawal amount or balance record in the m groups of evaluation data as unreasonable data.

[0175] The single deposit amount, single withdrawal amount and balance records in the m groups of evaluation data are all analyzed; if the single deposit amount is less than 0, the corresponding single deposit amount will be marked as unreasonable data; if the single deposit amount is greater than or equal to 0, the corresponding single deposit amount will not be marked as unreasonable data; if the single withdrawal amount is less than 0, the corresponding single withdrawal amount will be marked as unreasonable data; if the single withdrawal amount is greater than or equal to 0, the corresponding single withdrawal amount will not be marked as unreasonable data; if the balance record is less than 0, the corresponding balance record will be marked as unreasonable data; if the balance record is greater than or equal to 0, the corresponding balance record will not be marked as unreasonable data.

[0176] Step 203: Determine whether to mark a single deposit date or a single withdrawal date in the m groups of evaluation data as unreasonable data.

[0177] Mark the single deposit date and single withdrawal date in the m groups of evaluation data as date data, input the date data into the trained date analysis model to determine whether the date data is unreasonable.

[0178] The training process of the date analysis model includes:

[0179] The corresponding judgment results are set for a date data in advance, where a is an integer greater than 1, and the judgment results include reasonable dates and unreasonable dates. Different digital labels are set for both reasonable dates and unreasonable dates.

[0180] The numerical labels of the judgment results are marked as judgment labels, and the date data and the corresponding judgment labels are converted into a corresponding set of feature vectors.

[0181] Each group of feature vectors is used as the input of the date analysis model. The date analysis model takes a group of prediction judgment labels corresponding to each group of date data as output, and takes the actual judgment labels corresponding to each group of date data as the prediction targets, and the actual judgment labels are the above-mentioned pre-set digital labels of the judgment results corresponding to the date data; minimizing the sum of the prediction errors of all date data is used as the training target; the date analysis model is trained until the sum of the prediction errors reaches convergence and the training is stopped; the date analysis model is a deep neural network model.

[0182] The corresponding judgment result is obtained according to the predicted judgment label; if the judgment result is that the date is reasonable, the corresponding date data is not marked as unreasonable data; if the judgment result is that the date is unreasonable, the corresponding date data is marked as unreasonable data.

[0183] Step 204: Delete unreasonable data from the m groups of evaluation data.

[0184] Specifically, the method for performing quality assessment on the m groups of evaluation data after data cleaning includes:

[0185] The ID card number in the m groups of evaluation data is sent to the query server, which includes a social security query server, a household registration query server and an operator query server; among which, the social security query server queries according to the ID card number, generates the corresponding name and deposit unit, and provides feedback; the household registration query server queries according to the ID card number, generates the corresponding address, and provides feedback; the operator query server queries according to the ID card number, generates the corresponding telephone number, and provides feedback.

[0186] For the name, deposit unit, address and telephone number obtained from the feedback, the corresponding hash values ​​are calculated using a hash function and marked as comparison hash values; then the corresponding hash value is calculated for the deposit unit in the m groups of evaluation data using a hash function and also marked as a personal hash value.

[0187] Construct an analysis set with the comparison hash value corresponding to an ID card number; construct an evaluation set with the personal hash value corresponding to an ID card number; mark the name, deposit unit, address and telephone number as analysis data; compare the analysis set and evaluation set with the same corresponding ID card number, and compare each comparison hash value in the analysis set with the corresponding personal hash value in the evaluation set.

[0188] If the comparison hash value is the same as the corresponding personal hash value, the analysis data corresponding to the comparison hash value and the personal hash value are not marked as problematic data.

[0189] If the comparison hash value is different from the corresponding personal hash value, the analysis data corresponding to the comparison hash value and the personal hash value are marked as problematic data.

[0190] If the analysis data corresponding to each comparison hash value in an analysis set are all marked as problem data, then the corresponding analysis set will be marked as a problem set, and the ID number corresponding to the problem set will be marked as problem data; the phone number corresponding to the problem set will then be sent to the operator query server, and the operator query server will query based on the phone number, generate the corresponding name and provide feedback; the corresponding hash value of the fed-back name will be calculated using a hash function and marked as a secondary comparison hash value; the name corresponding to the problem set will be marked as a comparison name, and the comparison hash value corresponding to the comparison name will be compared with the secondary comparison hash value; if the comparison hash value is consistent with the secondary comparison hash value, the phone number and name corresponding to the problem set will not be marked as problem data; if the comparison hash value is inconsistent with the secondary comparison hash value, the phone number and name corresponding to the problem set will still be marked as problem data.

[0191] If the ID number in the assessment data is deleted, the name, deposit unit, address and telephone number in the corresponding assessment data that have not been deleted will be marked as problematic data.

[0192] Specifically, the warning instructions include high-level warning instructions and low-level warning instructions; the high-level warning instructions include first-level high-level warning instructions and second-level high-level warning instructions.

[0193] Specifically, the method for generating corresponding warning instructions for evaluation data for data cleaning includes:

[0194] If an identification card number or name in a set of evaluation data is marked as problematic data or deleted, a critical error instruction is generated.

[0195] If both the ID number and the name in a set of evaluation data are marked as problematic data or deleted at the same time, multiple key error instructions are generated.

[0196] If a transaction record, phone number or address in a set of evaluation data is marked as problematic data or deleted, a general error instruction is generated.

[0197] If transaction records, phone numbers or addresses in a set of evaluation data are all marked as problematic data or deleted, multiple general error instructions are generated.

[0198] If multiple critical error instructions are generated in a set of evaluation data, a second-level advanced warning instruction is generated.

[0199] If multiple critical error instructions are not generated in a set of evaluation data, but critical error instructions are generated, a first-level advanced warning instruction is generated.

[0200] If multiple critical error instructions and key error instructions are not generated in a set of evaluation data, but multiple general error instructions are generated, a first-level advanced warning instruction is generated.

[0201] If multiple critical error instructions, critical error instructions and multiple general error instructions are not generated in a set of evaluation data, but general error instructions are generated, a low-level warning instruction is generated.

[0202] Specifically, the evaluation data corresponding to the names with the same personal hash values ​​are analyzed to determine whether they are marked as duplicate data.

[0203] The evaluation data corresponding to names with the same personal hash value are marked as identical data.

[0204] If the ID card numbers in the same data in group b are all marked as duplicate data, then the names in the same data in group b are also marked as duplicate data, and b is an integer greater than 1.

[0205] If there are identity card numbers that are not marked as duplicate data in the b groups of identical data, the personal hash values ​​corresponding to each group of identical data are used as a set of test sets, that is, the test sets correspond to the identical data one-to-one; and the identical data corresponding to the identity card numbers that are not marked as duplicate data are marked as different data; digital labels are incrementally set for the test sets corresponding to the identical data and the test sets corresponding to the different data, and the digital labels corresponding to the test sets are marked as test labels, and the range of the test labels is [1, b′], b′=b.

[0206] The test labels corresponding to the same data and the test labels corresponding to a group of different data are taken as a total test set; each total test set is input into the trained probability prediction model in turn to predict the corresponding repetition probability.

[0207] The training process of the probabilistic prediction model includes:

[0208] The repetition probabilities corresponding to the total test sets of group b are collected in advance, and the total test sets and the corresponding repetition probabilities are converted into a corresponding set of feature vectors.

[0209] Each group of feature vectors is used as the input of the probability analysis model, and the probability analysis model takes a group of repetition probabilities corresponding to each group of total test sets as output, and takes the actual repetition probability corresponding to each group of total test sets as the prediction target, and the actual repetition probability is the pre-collected repetition probability corresponding to the total test set; minimizing the sum of prediction errors of all total test sets is used as the training goal; the probability analysis model is trained until the sum of prediction errors reaches convergence and the training is stopped; the probability analysis model is a deep neural network model.

[0210] Preset probability threshold P T .

[0211] The repetition probability P C With probability threshold P T for comparison.

[0212] If P C ≤P T , then the names in different data are not marked as duplicate data, and the names in the same data are marked as duplicate data.

[0213] If P C >P T , then the names in different data and the names in the same data are marked as duplicate data.

[0214] Example 4

[0215] See also Figure 4 As shown, according to another aspect of the present application, an electronic device 500 is also provided. The electronic device 500 may include one or more processors and one or more memories. The memories store computer readable codes, and when the computer readable codes are executed by one or more processors, the provident fund data quality assessment and graded warning method as described above may be executed.

[0216] The method or system according to the embodiment of the present application can also be used by Figure 4 The electronic device architecture shown in FIG. Figure 4As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, ROM 503, RAM 504, a communication port 505 connected to a network, an input / output 506, a hard disk 507, etc. The storage device in the electronic device 500, such as ROM 503 or hard disk 507, may store the provident fund data quality assessment and graded warning method provided in this application. Furthermore, the electronic device 500 may also include a user interface 508. Of course, Figure 4 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 4 One or more components of an electronic device are shown.

[0217] Example 5

[0218] See also Figure 5 As shown, a computer-readable storage medium 600 according to an embodiment of the present application is shown. Computer-readable instructions are stored on the computer-readable storage medium 600. When the computer-readable instructions are executed by the processor, the provident fund data quality assessment and graded early warning method according to the embodiment of the present application described with reference to the above figures can be executed. The storage medium 600 includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0219] In addition, according to the implementation of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided in the present application, such as: Provident Fund Data Quality Assessment and Grading Early Warning Method. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed.

[0220] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0221] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. Provident fund data quality assessment and graded early warning method, characterized in that: include: Collect m groups of assessment data from the provident fund system, where m is an integer greater than 1; The evaluation data includes at least personal information, transaction records and balance records; The personal information includes the name, ID number, telephone number and address of the provident fund account holder; The transaction records include deposit records and withdrawal records; The deposit record includes the single deposit amount, single deposit date and deposit unit; Perform data cleaning on m groups of evaluation data; The step of performing data cleaning on the m groups of evaluation data comprises: Step 1: Delete duplicate data in m groups of evaluation data; Step 2: Delete unreasonable data from the m groups of evaluation data; The method for deleting duplicate data in m groups of evaluation data comprises: For each data in each group of personal information in the m groups of evaluation data, a corresponding hash value is calculated using a hash function and marked as a personal hash value; each personal hash value is compared to determine whether there is an identical personal hash value; if there is an identical personal hash value, the data with the same personal hash value is marked as duplicate data, one duplicate data among the n duplicate data with the same personal hash value is retained, and the remaining n-1 duplicate data are deleted from the m groups of evaluation data, where n is an integer greater than 1; If the name and ID number in the personal information in one set of evaluation data are deleted, the corresponding evaluation data will be deleted from the m sets of evaluation data; Analyze the evaluation data corresponding to names with the same personal hash value to determine whether to mark them as duplicate data; Mark the evaluation data corresponding to names with the same personal hash value as the same data; If the ID card numbers in the same data in group b are all marked as duplicate data, then the names in the same data in group b are also marked as duplicate data, where b is an integer greater than 1; If there are ID card numbers that are not marked as duplicate data in the b groups of identical data, the personal hash values ​​corresponding to each group of identical data are used as a set of test sets, that is, the test sets correspond to the identical data one by one; and the identical data corresponding to the ID card numbers that are not marked as duplicate data are marked as different data; digital labels are set incrementally for the test sets corresponding to the identical data and the test sets corresponding to the different data, and the digital labels corresponding to the test sets are marked as test labels, and the range of the test labels is [1, b′], b′=b; The test labels corresponding to the same data and the test labels corresponding to a group of different data are taken as a group of total test sets; each group of total test sets is input into the trained probability prediction model in turn to predict the corresponding repetition probability; The training process of the probabilistic prediction model includes: Pre-collect the repetition probabilities corresponding to the total test set of group b, and convert the total test set and the corresponding repetition probabilities into a corresponding set of feature vectors; Each group of feature vectors is used as the input of the probability analysis model. The probability analysis model uses a group of repetition probabilities corresponding to each group of total test sets as output, and uses the actual repetition probability corresponding to each group of total test sets as the prediction target, where the actual repetition probability is the pre-collected repetition probability corresponding to the total test set; minimizing the sum of prediction errors of all total test sets is used as the training target; the probability analysis model is trained until the sum of prediction errors reaches convergence and the training is stopped; the probability analysis model is a deep neural network model; Preset probability threshold P T ; The probability of repetition P C With probability threshold P T Make a comparison; If P C ≤P T , then the names in different data are not marked as duplicate data, and the names in the same data are marked as duplicate data; If P C >P T , then the names in different data and the names in the same data are marked as duplicate data; Perform quality assessment on the m groups of evaluation data after data cleaning and automatically identify quality issues; The method for performing quality assessment on the m groups of evaluation data after data cleaning includes: The ID card number in the m group of evaluation data is sent to the query server, which includes a social security query server, a household registration query server and an operator query server; wherein the social security query server queries according to the ID card number, generates the corresponding name and deposit unit, and gives feedback; the household registration query server queries according to the ID card number, generates the corresponding address, and gives feedback; the operator query server queries according to the ID card number, generates the corresponding telephone number, and gives feedback; For the name, deposit unit, address and telephone number obtained from the feedback, the corresponding hash values ​​are calculated using a hash function and marked as comparison hash values; the corresponding hash values ​​of the deposit units in the m groups of evaluation data are then calculated using a hash function and also marked as personal hash values; Construct an analysis set from the comparison hash value corresponding to an ID card number; construct an evaluation set from the personal hash value corresponding to an ID card number; mark the name, deposit unit, address and telephone number as analysis data; compare the analysis set and evaluation set with the same corresponding ID card number, and compare each comparison hash value in the analysis set with the corresponding personal hash value in the evaluation set; If the comparison hash value is the same as the corresponding personal hash value, the analysis data corresponding to the comparison hash value and the personal hash value is not marked as problematic data; If the comparison hash value is different from the corresponding personal hash value, the analysis data corresponding to the comparison hash value and the personal hash value are marked as problematic data; If the analysis data corresponding to each comparison hash value in an analysis set are all marked as problem data, then the corresponding analysis set is marked as a problem set, and the ID number corresponding to the problem set is marked as problem data; then the phone number corresponding to the problem set is sent to the operator query server, and the operator query server queries based on the phone number, generates the corresponding name and gives feedback; the hash value corresponding to the fed-back name is calculated using a hash function and marked as a secondary comparison hash value; the name corresponding to the problem set is marked as a comparison name, and the comparison hash value corresponding to the comparison name is compared with the secondary comparison hash value; if the comparison hash value is consistent with the secondary comparison hash value, the phone number and name corresponding to the problem set are not marked as problem data; if the comparison hash value is inconsistent with the secondary comparison hash value, the phone number and name corresponding to the problem set are still marked as problem data; If the ID number in the assessment data is deleted, the name, deposit unit, address and telephone number in the corresponding assessment data that have not been deleted will be marked as problematic data; Generate corresponding early warning instructions for data cleaning and evaluation data with quality issues identified.

2. The method for providing housing fund data quality assessment and graded early warning according to claim 1 is characterized in that: The withdrawal record includes a single withdrawal amount and a single withdrawal date; The balance record is the balance information in the provident fund account; A set of assessment data corresponds to a provident fund account in the provident fund system.

3. The method for providing housing fund data quality assessment and graded early warning according to claim 2 is characterized in that: The step of deleting unreasonable data in the m groups of evaluation data comprises: Step 201: Determine whether to mark the ID card number or telephone number in each set of personal information in the m sets of evaluation data as unreasonable data; For the ID card number and telephone number in each group of personal information in the m groups of evaluation data, a string function is used to obtain the corresponding string length, and the string length corresponding to the ID card number is marked as the ID card length, and the string length corresponding to the telephone number is marked as the telephone length; the ID card length threshold and the telephone length threshold are preset, and each ID card length is compared with the ID card length threshold. If the ID card length is inconsistent with the ID card length threshold, the corresponding ID card number is marked as unreasonable data; if the ID card length is consistent with the ID card length threshold, the corresponding ID card number is not marked as unreasonable data; each telephone length is compared with the telephone length threshold. If the telephone length is inconsistent with the telephone length threshold, the corresponding telephone number is marked as unreasonable data; if the telephone length is consistent with the telephone length threshold, the corresponding telephone number is not marked as unreasonable data; Step 202: Determine whether to mark the single deposit amount, single withdrawal amount or balance record in the m groups of evaluation data as unreasonable data; The single deposit amount, single withdrawal amount and balance record in the m group of evaluation data are analyzed; if the single deposit amount is less than 0, the corresponding single deposit amount is marked as unreasonable data; if the single deposit amount is greater than or equal to 0, the corresponding single deposit amount is not marked as unreasonable data; if the single withdrawal amount is less than 0, the corresponding single withdrawal amount is marked as unreasonable data; if the single withdrawal amount is greater than or equal to 0, the corresponding single withdrawal amount is not marked as unreasonable data; if the balance record is less than 0, the corresponding balance record is marked as unreasonable data; if the balance record is greater than or equal to 0, the corresponding balance record is not marked as unreasonable data; Step 203: Determine whether to mark a single deposit date or a single withdrawal date in the m groups of evaluation data as unreasonable data; Mark the single deposit date and single withdrawal date in the m group of evaluation data as date data, input the date data into the trained date analysis model, and determine whether the date data is unreasonable; The training process of the date analysis model includes: Pre-set corresponding judgment results for a date data, where a is an integer greater than 1, and the judgment results include reasonable dates and unreasonable dates, and different digital labels are set for reasonable dates and unreasonable dates; The digital labels of the judgment results are marked as judgment labels, and the date data and the corresponding judgment labels are converted into a corresponding set of feature vectors; Each group of feature vectors is used as the input of the date analysis model. The date analysis model uses a group of prediction judgment labels corresponding to each group of date data as output, and uses the actual judgment labels corresponding to each group of date data as the prediction target, and the actual judgment labels are the digital labels of the judgment results corresponding to the date data pre-set above; minimizing the sum of the prediction errors of all date data is used as the training target; the date analysis model is trained until the sum of the prediction errors reaches convergence and the training is stopped; the date analysis model is a deep neural network model; Obtain the corresponding judgment result according to the predicted judgment label; if the judgment result is that the date is reasonable, the corresponding date data is not marked as unreasonable data; if the judgment result is that the date is unreasonable, the corresponding date data is marked as unreasonable data; Step 204: Delete unreasonable data from the m groups of evaluation data.

4. The method for providing housing fund data quality assessment and graded early warning according to claim 3 is characterized in that: The warning instructions include high-level warning instructions and low-level warning instructions; the high-level warning instructions include first-level high-level warning instructions and second-level high-level warning instructions.

5. The method for providing housing fund data quality assessment and graded early warning according to claim 4 is characterized in that: The method for generating corresponding early warning instructions for evaluation data for data cleaning includes: If an ID number or name in a set of evaluation data is marked as problematic data or deleted, a critical error instruction is generated; If both the ID number and the name in a set of evaluation data are marked as problematic data or deleted at the same time, a multi-key error instruction is generated; If a transaction record, phone number or address in a set of evaluation data is marked as problematic data or deleted, a general error instruction is generated; If transaction records, phone numbers or addresses in a set of evaluation data are all marked as problematic data or deleted, multiple general error instructions are generated; If multiple critical error instructions are generated in a set of evaluation data, a second-level advanced warning instruction is generated; If multiple critical error instructions are not generated in a set of evaluation data, but critical error instructions are generated, a first-level advanced warning instruction is generated; If multiple critical error instructions and key error instructions are not generated in a set of evaluation data, but multiple general error instructions are generated, a first-level advanced warning instruction is generated; If multiple critical error instructions, critical error instructions and multiple general error instructions are not generated in a set of evaluation data, but general error instructions are generated, a low-level warning instruction is generated.

6. A provident fund data quality assessment and graded early warning system, implementing the provident fund data quality assessment and graded early warning method according to any one of claims 1 to 5, characterized in that: include: A data collection module is used to collect m groups of evaluation data from the provident fund system, where m is an integer greater than 1; A data preprocessing module is used to clean the m groups of evaluation data; The data quality assessment module is used to perform quality assessment on the m groups of evaluation data after data cleaning and automatically identify quality problems; The early warning module is used to generate corresponding early warning instructions for data cleaning and evaluation data with quality problems identified.