A PSI-based data anomaly intelligent monitoring method and an intelligent monitoring device

Abstract

The application provides a PSI-based data anomaly intelligent monitoring method, device, system and recording medium, and the method comprises the following steps: obtaining a data set to be monitored, the data set comprising a plurality of data entries, each data entry comprising at least one data item; calculating a stability index PSI of the at least one data item of all data entries of the data set; and when the PSI is outside a specific threshold range, performing alarm processing. By calculating the stability index PSI of the at least one data item of all data entries of the data set, when the PSI is outside a specific threshold range, alarm processing is performed. When the credit user data changes, the data anomaly can be easily understood, so that various statistical tasks can be completed without manual intervention. Therefore, the system can greatly improve the statistical efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of financial credit services, and in particular to an intelligent monitoring method and device based on a population stability index (PSI). BACKGROUND

[0002] In the prior art, personal loan or credit decision-making requires rating the customer first, and then calculating the loan or credit limit. Currently, a common method is to construct a personal credit scorecard through a credit model, to evaluate the personal credit of a user based on information obtained from banks and other institutions, and to determine the loan limit according to the personal credit evaluation of the user.

[0003] The PSI is a degree of change of a feature or a score over time. For example, the income of a user. If we want to observe the degree of change of the distribution of the income of a user over time, for example, the percentage of more than 20,000 is 35%, the percentage of 10,000 to 20,000 is 50%, and the percentage of less than 10,000 is 15%. When observing the change of the percentages over time, it may be that before May, the percentages are 35%, 50%, and 15%, but now they have changed to 15%, 50%, and 35%. Obviously, this indicates that the percentage of income below 10,000 has increased, which may indicate that the quality of the user group has deteriorated, but such a change is generally difficult to quantify.

[0004] Therefore, there is an urgent need for an intelligent monitoring method and device for data anomalies, which can automatically establish and verify credit scoring. SUMMARY

[0005] The present application solves the problem of real-time and effective monitoring of user data in personal loan or credit decision-making.

[0006] To solve the above technical problems, the first aspect of the present application provides an intelligent monitoring method for data anomalies based on PSI, which comprises the following steps:

[0007] Obtaining a data set to be monitored for data anomalies, the data comprising a plurality of data entries, each data entry comprising at least one data item;

[0008] Calculating the stability index PSI of at least one data item of all data entries of the data set;

[0009] When the PSI is outside a specific threshold range, an alarm is triggered.

[0010] According to a preferred embodiment of the present application, the step of calculating the stability index PSI of at least one data item of all data entries of the data set comprises:

[0011] According to the value of the at least one data item included in each data entry, the data entries are binned, and the proportion of the number of data entries in each bin to the total number of data entries is calculated.

[0012] According to a preferred embodiment of the present application, the step of calculating the stability index PSI of the at least one data item of all data entries of the data set further comprises: comparing the proportion of each bin with a standard proportion, and calculating the PSI.

[0013] According to a preferred embodiment of the present application, the standard proportion is the proportion of the same bin in a historical data set of the same type as the data set.

[0014] According to a preferred embodiment of the present application, the formula for calculating the PSI is:

[0015] wherein A i represents the standard proportion of data entries in the i-th bin, B i represents the proportion of data entries in the i-th bin in the historical data set of the same type, and n is the number of samples.

[0016] According to a preferred embodiment of the present application, the data set comprises: a data set of credit user data.

[0017] According to a preferred embodiment of the present application, the at least one data item comprises: any one of income, liability, valuable securities, and commercial insurance.

[0018] According to a preferred embodiment of the present application, when the PSI is outside a certain threshold range, the step of alarm processing comprises: displaying the name, data, bin, and proportion of the data item.

[0019] The second aspect of the present application proposes an intelligent data anomaly monitoring device based on PSI, comprising:

[0020] An acquisition module is configured to acquire a data set to be monitored for data anomaly, the data set comprising a plurality of data entries, each data entry comprising at least one data item;

[0021] A calculation module is configured to calculate a stability index PSI of the at least one data item of all data entries of the data set.

[0022] An alarm processing module is configured to perform alarm processing when the PSI is outside a certain threshold range.

[0023] According to a preferred embodiment of the present application, the computing module is further configured to bin each data entry according to the value of the at least one data item included in the data entry, and calculate the proportion of the number of data entries in each bin to the total number of data entries.

[0024] According to a preferred embodiment of the present application, the computing module is further configured to compare the proportion of each bin to a standard proportion, and calculate the PSI.

[0025] According to a preferred embodiment of the present application, the standard proportion is the proportion of the same bin in a historical data set of the same type as the data set.

[0026] According to a preferred embodiment of the present application, the formula for calculating the PSI is:

[0027] wherein A i represents the standard proportion of data entries in the i-th bin, B i represents the proportion of data entries in the i-th bin in the historical data set of the same type, and n is the number of sampling times.

[0028] According to a preferred embodiment of the present application, the data set comprises a data set of credit user data.

[0029] According to a preferred embodiment of the present application, the at least one data item comprises any one of income, liability, valuable securities, and commercial insurance.

[0030] According to a preferred embodiment of the present application, the alarm processing module is further configured to, when the PSI is outside a specific threshold range, perform the step of alarm processing comprising displaying the name, data, bin, and proportion of the data item.

[0031] A third aspect of the present application provides a PSI-based data anomaly intelligent monitoring system, comprising:

[0032] a memory for storing computer executable programs;

[0033] a data processing device for reading the computer executable programs in the memory to execute the PSI-based data anomaly intelligent monitoring method.

[0034] A fourth aspect of the present application provides a computer readable medium for storing computer readable programs, wherein the computer readable programs are configured to execute the PSI-based data anomaly intelligent monitoring method.

[0035] The application provides a PSI-based data anomaly intelligent monitoring method, which comprises the following steps: S1, acquiring a data set to be monitored, wherein the data set comprises a plurality of data entries, and each data entry comprises at least one data item; S2, calculating a stability index PSI of the at least one data item of all data entries in the data set; and S3, performing an alarm process when the PSI is located outside a specific threshold range. When the credit user data changes, the data anomaly can be easily understood, so that various statistical tasks can be completed without manual intervention. Therefore, the system can greatly improve the statistical efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 FIG. 1 is a flowchart of a PSI-based data anomaly intelligent monitoring method according to the application;

[0037] Figure 2 FIG. 2 is a schematic diagram of a PSI-based data anomaly intelligent monitoring architecture according to the application;

[0038] Figure 3 FIG. 3 is a schematic diagram of a PSI-based data anomaly intelligent monitoring system framework according to the application. DETAILED DESCRIPTION

[0039] In order to make the objects, technical solutions and advantages of the application clearer, the application will be further described in detail below with reference to the embodiments and the accompanying drawings.

[0040] Figure 1 FIG. 1 is a flowchart of a PSI-based data anomaly intelligent monitoring method according to the application, as shown in the figure, the method of the application has the following steps: Figure 1

[0041] S1, acquiring a data set to be monitored, wherein the data set comprises a plurality of data entries, and each data entry comprises at least one data item; preferably, the data set comprises a credit user data set; more preferably, the at least one data item comprises any one of income, liability, valuable securities and commercial insurance.

[0042] In the preferred embodiment, the step of calculating the stability index PSI of the at least one data item of all data entries in the data set in step S1 comprises: grading each data entry according to the numerical value of the at least one data item included in the data entry, and calculating the proportion of the number of data entries in each grade to the total number of data entries.

[0043] S2, calculating the stability index PSI of the at least one data item of all data entries in the data set.

[0044] ​In a preferred embodiment, the step of calculating the stability index PSI of at least one data item of all data items of the data set in step S2 further comprises: comparing the proportion of each sub-range with a standard proportion, and calculating the PSI.

[0045] Preferably, the standard proportion is the proportion of the same sub-range in a historical data set of the same type.

[0046] A reference system can be selected, and a percentage (35%, 50%, 15%) of the reference system can be calculated based on the data in the period from May to June, which is assumed to be stable. Then, a percentage of today can be calculated, such as 15%, 50%, and 35%. More preferably, the formula for calculating the PSI is:

[0047] wherein A i represents the standard proportion of data items in the i-th sub-range, B i represents the proportion of data items in the i-th sub-range in a historical data set of the same type, and n is the sampling number.

[0048] The values of each sub-range, A1, A2, and A3, can be calculated, and A1+A2+A3=0.26 is assumed. If the PSI is greater than 0.25, the deviation is relatively large, if the PSI is between 0.02 and 0.25, the deviation is a little, and if the PSI is less than 0.02, the deviation is basically small.

[0049] S3, when the PSI is outside a specific threshold range, an alarm is processed.

[0050] In this way, a number can be calculated for a certain field every day. We now select those fields with high importance in the model, and a calculation is performed every day.

[0051] According to a preferred embodiment of the present application, when the PSI is outside a specific threshold range, the step of processing an alarm comprises displaying the name, data, sub-range, and proportion of the data item. For example, a field usually accounts for 1% of -1, but suddenly accounts for 25%, and the PSI is 0.76, which is an alarm for all data sources.

[0052] The present application provides a PSI-based data anomaly intelligent monitoring method, which calculates the stability index PSI of at least one data item of all data items of a data set, and processes an alarm when the PSI is outside a specific threshold range. When the credit user data changes, the data anomaly can be easily understood. Thus, various statistical tasks can be completed without manual intervention. Therefore, the method can greatly improve the statistical efficiency and improve the accuracy of credit scoring.

[0053] The application provides a PSI-based data anomaly intelligent monitoring method, which can automatically establish and verify credit scoring.

[0054] The application provides a PSI-based data anomaly intelligent monitoring method, which quantifies the risk of a customer, thereby helping a loan institution to make a decision on whether to loan the customer; compared with a traditional manual decision, the PSI-based data anomaly intelligent monitoring method has higher accuracy, lower cost and less time consumption, and thus is beneficial to risk management of the loan institution.

[0055] Figure 3 The application provides a PSI-based data anomaly intelligent monitoring device architecture schematic diagram. As shown in the figure, the device architecture schematic diagram comprises: Figure 3

[0056] The acquisition module 201 is configured to acquire a data set to be monitored for data anomaly, wherein the data set comprises a plurality of data entries, and each data entry comprises at least one data item.

[0057] Preferably, the data set comprises a credit user data set. More preferably, the at least one data item comprises any one of income, liability, valuable securities and commercial insurance.

[0058] The calculation module 202 is configured to calculate a stability index PSI of the at least one data item of all data entries of the data set. Preferably, the calculation module 202 is further configured to classify each data entry according to the value of the at least one data item included in the data entry, and calculate the proportion of the number of data entries in each classification to the total number of data entries.

[0059] More preferably, the calculation module 202 is further configured to compare the proportion of each classification with a standard proportion, and calculate the PSI. According to a preferred embodiment of the application, the formula for calculating the PSI is as follows:

[0060] Wherein, A i represents the standard proportion of data entries in the i-th classification, B i represents the proportion of data entries in the i-th classification in the historical data set of the same type, and n is the sampling number.

[0061] Preferably, the standard proportion is the proportion of the same classification in the historical data set of the same type of the data set.

[0062] The alarm processing module 203 is configured to perform alarm processing when the PSI is located outside a specific threshold range. ​

[0063] Preferably, the alarm processing module 203 is further configured to perform the step of alarm processing when the PSI is outside a specific threshold range, and the step of displaying the name, data, bin, and proportion of the data item.

[0064] Figure 3 is a schematic diagram of a PSI-based data anomaly intelligent monitoring system framework provided by the present application. As shown in the figure, it comprises: Figure 3

[0065] a memory 301 configured to store computer executable programs;

[0066] a data processing device 302 configured to read the computer executable programs in the memory to execute the PSI-based data anomaly intelligent monitoring method.

[0067] Further, the present application also provides a computer readable medium configured to store a computer readable program, and the computer readable program is configured to execute the PSI-based data anomaly intelligent monitoring method.

[0068] It should be understood that, in order to simplify the present application and help those skilled in the art to understand various aspects of the present application, in the above description of the exemplary embodiments of the present application, various features of the present application are sometimes described in a single embodiment, or described with reference to a single figure. However, the present application should not be interpreted as the features included in the exemplary embodiments are all necessary technical features of the patent claims of the present application.

[0069] It should be understood that the modules, units, components, etc. included in the device of one embodiment of the present application can be adaptively changed to be arranged in a device different from the embodiment. Different modules, units or components included in the device of the embodiment can be combined into one module, unit or component, or divided into multiple sub-modules, sub-units or sub-components. The modules, units or components in the embodiment of the present application can be implemented in a hardware manner, or implemented in a software manner running on one or more processors, or implemented in a combination thereof.

[0070] The above described specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only for specific embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.​

Claims

1. A real-time intelligent monitoring method based on PSI data anomaly, characterized in that: The method comprises the following steps: Obtaining a data set to be monitored for data anomaly, the data comprising a plurality of data entries, each data entry comprising at least one data item; Calculating a stability index PSI of the at least one data item of all data entries of the data set, and using the stability index PSI to represent a quality change of a user group; filing each data entry according to a value of the at least one data item included in the data entry, calculating a proportion of a number of data entries in each file to a total number of data entries, and comparing the proportion in each file to a standard proportion to determine the PSI, the standard proportion being a proportion of a same file in a historical data set of the same type as the data set; a formula for calculating the PSI being as follows: wherein A i represents the standard proportion of data entries under the i-th sub-range, B i represents the proportion of data entries under the i-th sub-range in the historical data of the same kind, and n is the sampling number; the calculation of the PSI includes the calculation of the PSI value of the data of a certain field data every day, and the field data is selected from the field data with high importance ranking in the model; When the user data change causes the PSI to be outside a specific threshold range, performing alarm processing to quickly find data anomaly, the alarm processing comprising displaying a name, data, file and proportion of the data item.

2. A real-time intelligent monitoring device based on PSI data anomaly, characterized in that, The method comprises: An obtaining module, configured to obtain a data set to be monitored for data anomaly, the data comprising a plurality of data entries, each data entry comprising at least one data item; A calculating module, configured to calculate a stability index PSI of the at least one data item of all data entries of the data set, and using the stability index PSI to represent a quality change of a user group; filing each data entry according to a value of the at least one data item included in the data entry, calculating a proportion of a number of data entries in each file to a total number of data entries, and comparing the proportion in each file to a standard proportion to determine the PSI, the standard proportion being a proportion of a same file in a historical data set of the same type as the data set; a formula for calculating the PSI being as follows: wherein A i represents the standard proportion of data entries under the i-th sub-range, B i represents the proportion of data entries under the i-th sub-range in the historical data of the same kind, and n is the sampling number; the calculation of the PSI includes the calculation of the PSI value of the data of a certain field data every day, and the field data is selected from the field data with high importance ranking in the model; An alarm processing module, configured to, when the user data change causes the PSI to be outside a specific threshold range, perform alarm processing to quickly find data anomaly, the alarm processing comprising displaying a name, data, file and proportion of the data item.

3. A real-time intelligent monitoring system based on PSI data anomaly, characterized in that, The method comprises: A memory, configured to store a computer executable program; A data processing device, configured to read the computer executable program in the memory to execute the method in claim 1.

4. A computer readable medium for storing a computer readable program, characterized in that, The computer readable program is configured to execute the method in claim 1. The computer readable program is configured to execute the method in claim 1.

Images