A real-time monitoring and early warning method and system for a three-party data information platform

Abstract

The present invention provides a real-time monitoring and early warning method and system for a third-party data information platform, comprising: collecting data source scores corresponding to different client organizations, scoring products, model versions, and time point information of the third-party data platform; determining the month-on-month increase or decrease and the rate of change of the absolute value of each preset indicator under a preset dimension based on the historical data source scores, and establishing early warning rules based on the month-on-month increase or decrease and the rate of change of the absolute value; determining whether data anomalies appear in the data source scores according to the early warning rules, and pushing early warnings after data anomalies appear; obtaining early warning results under the early warning rules within a preset time, and analyzing the early warning results to ensure the comprehensiveness of the establishment of early warning rules, and facilitating efficient and timely early warning of faults by performing abnormal analysis on third-party data according to the early warning rules, quickly making response plans, optimizing the early warning rules, and improving the accuracy of the early warning rules.

Description

Technical Field

[0001] The present invention relates to the technical field of data information monitoring, and in particular to a real-time monitoring and early warning method and system for a third-party data information platform. Background Art

[0002] In the era of big data, data resources are an important infrastructure for enterprise management. In the field of financial credit, external third-party data is an important data resource for financial institutions. The reliance of their scoring data products on externally purchased data reflects the necessity of monitoring and early warning of third-party data. In order to avoid production failures of third-party data, which may cause errors or paralysis of the real-time scoring data production system due to delayed response and result in economic losses, the calling institution needs to establish a sound real-time monitoring and early warning mechanism to effectively and timely warn of externally purchased data failures, quickly make response plans, and improve operational efficiency. Summary of the invention

[0003] The present invention provides a real-time monitoring and early warning method and system for a third-party data information platform, establishes an accurate and complete real-time monitoring and early warning mechanism, and improves operational efficiency.

[0004] A real-time monitoring and early warning method for a third-party data information platform, comprising:

[0005] Step 1: Collect data source scores from third-party data platforms for different client organizations, scoring products, model versions, and time point information;

[0006] Step 2: Based on the historical data source scores, determine the month-on-month increase or decrease and the absolute value change rate of each preset indicator under the preset dimension, and establish an early warning rule based on the month-on-month increase or decrease and the absolute value change rate;

[0007] Step 3: According to the warning rule, determine whether the data source score has data anomalies, and push warnings when data anomalies occur;

[0008] Step 4: Obtain the warning results under the warning rules within a preset time, analyze the warning results, and optimize the warning rules according to the analysis results.

[0009] Preferably, in step 1, collecting the data source scores corresponding to different client organizations, scoring products, model versions, and time point information from the third-party data platform includes:

[0010] Obtain scoring rules for different client organizations, scoring products, model versions, and time point information on third-party data platforms;

[0011] Based on the scoring rules, the data sources corresponding to the different client organizations, scoring products, model versions, and time point information are scored to obtain data source scores.

[0012] Preferably, after scoring the data sources corresponding to the different client organizations, scoring products, model versions, and time point information, the method further includes:

[0013] Perform anomaly detection on the scored data sources and extract the target data sources with missing scores;

[0014] According to the mapping relationship, the target data source is filled and processed to obtain a score of the target data source.

[0015] Preferably, in step 2, based on the historical data source score, determining the month-on-month increase or decrease and the absolute value change rate of each preset indicator under the preset dimension includes:

[0016] Based on the preset dimensions, the historical data source scores are divided to obtain relevant data source scores corresponding to each preset dimension;

[0017] Under the preset dimension, the scores of the relevant data sources are analyzed according to the preset indicators to determine the month-on-month increase or decrease and the rate of change of the absolute value under each preset indicator.

[0018] Preferably, the scores of the relevant data sources are analyzed according to the preset indicators to determine the month-on-month increase or decrease and the absolute value change rate under each preset indicator, including:

[0019] The preset indicators include the rating call volume, the average rating, the rating score availability rate and the rating abnormality request rate;

[0020] The scoring call volume is divided into call volume and effective call volume. The call volume is determined by the sum of all calls of each client institution to different scoring products and model versions according to the relevant data source scores; the effective call volume is the sum of normal calls determined by making special calls;

[0021] The score is divided into the average of the scores of the normal data sources in the scores of the relevant data sources;

[0022] The scoring rate is the ratio between the effective call volume and the call volume;

[0023] The scoring abnormal request rate is the ratio between the number of abnormal calls with invalid scoring and scoring timeout and the number of calls;

[0024] Set a grading period and set the interval time under the grading period;

[0025] Based on the scoring cycle and interval time, the month-on-month increase or decrease and the absolute value change rate of each preset indicator are calculated.

[0026] Preferably, in step 2, based on the month-on-month increase or decrease and the rate of change of the absolute value, an early warning rule is established, including:

[0027] Obtaining warning information from historical data sources, and determining the reference month-on-month increase / decrease and reference change rate for preliminarily determining that data anomalies occur based on the warning information from the month-on-month increase / decrease and the change rate of the absolute value;

[0028] Based on the characteristics of the preset dimension, determining the importance coefficients of different preset indicators to the preset dimension;

[0029] Based on the historical monitoring information of the preset dimension, determining a first warning threshold value of each preset indicator under the preset dimension;

[0030] Based on the importance coefficient, the first warning threshold is corrected to obtain a second warning threshold;

[0031] Based on the characteristics of the preset dimensions, determining the correlation characteristics between the preset dimensions, and based on the correlation characteristics, determining the correlation relationship between different preset dimensions and the same preset indicator;

[0032] Based on the association relationship, determine the value constraint relationship between the same preset indicator under different preset dimensions;

[0033] Based on the value constraint relationship, determining the value difference range of the different preset dimensions between the second warning thresholds of the same preset indicator;

[0034] Extracting the second warning threshold to be adjusted whose second warning threshold does not meet the value difference range, and adjusting the second warning threshold to be adjusted based on the correlation relationship between the preset indicators corresponding to the second warning threshold to be adjusted and the value difference range;

[0035] Obtain the preset month-on-month increase or decrease and the preset change rate determined by the adjusted second warning threshold;

[0036] Determine whether the difference between the preset month-on-month increase / decrease and the preset change rate and the reference month-on-month increase / decrease and the reference change rate is within a preset range;

[0037] If yes, the preset month-on-month increase / decrease and the preset change rate are used as threshold values ​​of the warning rule to establish the warning rule;

[0038] Otherwise, a new value difference range is re-determined, and the second warning threshold to be adjusted is adjusted until the preset month-on-month increase or decrease and the preset change rate determined by the adjusted second warning threshold meet the requirements.

[0039] Preferably, in step 3, determining whether data anomalies occur in the data source score according to the early warning rule includes:

[0040] Based on the warning rules, determine the month-on-month increase / decrease threshold and the absolute value change rate threshold of the preset indicator under each preset dimension;

[0041] Based on the month-on-month increase / decrease threshold and the absolute value change rate threshold, the monitored month-on-month increase / decrease and the absolute value change rate are graded, and the early warning sub-content corresponding to the preset indicator under each preset dimension is determined at each grade;

[0042] Establish a data warning sub-content table based on the month-on-month increase / decrease threshold, the absolute value change rate threshold, and the warning sub-content;

[0043] Performing a correlation analysis on any two warning sub-contents to determine whether there is a correlation between the two warning sub-contents; if there is a correlation, performing a combined analysis on the two warning sub-contents to determine the relationship type corresponding to the two warning sub-contents;

[0044] Based on the relationship type, merging the two warning sub-contents to obtain warning content;

[0045] Based on the warning content, the data warning sub-content table is updated to obtain a warning content table;

[0046] Obtain the actual month-on-month increase or decrease and actual change rate of the data source score under each preset dimension and preset indicator;

[0047] Matching the actual month-on-month increase / decrease and the actual change rate with the warning content table to determine whether the match is successful;

[0048] If so, determine that data anomalies occur in the data source score, and determine the actual warning content based on the matching result;

[0049] Otherwise, it is determined that no data anomaly occurs in the data source scoring.

[0050] Preferably, in step 3, after data anomaly occurs, an early warning is pushed, including:

[0051] Determine the actual warning content according to the matching result, and generate an alarm email according to the actual warning content;

[0052] The alarm email is pushed to the mailbox of the monitoring personnel.

[0053] Preferably, in step 4, obtaining the warning result under the warning rule within a preset time, analyzing the warning result, and optimizing the warning rule according to the analysis result includes:

[0054] Based on the warning results, determine the number of warnings within a preset time, and the data investigation results corresponding to each warning;

[0055] Determine the accurate value of each warning based on the warning results and data investigation results;

[0056] Based on the number of warnings and the accuracy value, an analysis result is obtained, and a warning result in the analysis result in which the accuracy value is greater than a preset accuracy value is taken as a positive sample, and other warning results are taken as negative samples;

[0057] Determining the accuracy of the warning rule based on the positive samples and the negative samples;

[0058] Using the warning result corresponding to the positive sample as an iterative rule group, and determining the number of iterations according to the accuracy rate;

[0059] Based on the iterative rule group, the early warning rule is iteratively optimized according to the number of iterations to obtain an optimized early warning rule.

[0060] A real-time monitoring and early warning system for a three-party data information platform, comprising:

[0061] The data source module is used to collect data source scores corresponding to different client organizations, scoring products, model versions, and time point information from third-party data platforms;

[0062] A rule design module is used to determine the month-on-month increase or decrease and the absolute value change rate of each preset indicator under the preset dimension based on the historical data source score, and establish an early warning rule based on the month-on-month increase or decrease and the absolute value change rate;

[0063] An early warning push module is used to determine whether data anomalies occur in the data source score according to the early warning rule, and push an early warning if data anomalies occur;

[0064] The rule optimization module is used to obtain the warning results under the warning rules within a preset time, analyze the warning results, and optimize the warning rules according to the analysis results.

[0065] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0066] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0068] Figure 1 It is a flow chart of a real-time monitoring and early warning method of a third-party data information platform in an embodiment of the present invention;

[0069] Figure 2 Another flow chart of a real-time monitoring and early warning method of a third-party data information platform in an embodiment of the present invention;

[0070] Figure 3 It is a structural diagram of a real-time monitoring and early warning system of a third-party data information platform in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0072] Example 1

[0073] The embodiment of the present invention provides a real-time monitoring and early warning method for a third-party data information platform, such as Figure 1 As shown, including:

[0074] Step 1: Collect data source scores from third-party data platforms for different client organizations, scoring products, model versions, and time point information;

[0075] Step 2: Based on the historical data source scores, determine the month-on-month increase or decrease and the absolute value change rate of each preset indicator under the preset dimension, and establish an early warning rule based on the month-on-month increase or decrease and the absolute value change rate;

[0076] Step 3: According to the warning rule, determine whether the data source score has data anomalies, and push warnings when data anomalies occur;

[0077] Step 4: Obtain the warning results under the warning rules within a preset time, analyze the warning results, and optimize the warning rules according to the analysis results.

[0078] In this embodiment, the data source scoring is the data scoring corresponding to the different client organizations, scoring products, model versions, and time point information.

[0079] In this embodiment, the preset dimensions are, for example, customer organization units, product service code units, product version units, and third-party data platform units.

[0080] In this embodiment, the preset dimensions include customer organization units, product service code units, product version units, third-party data platform units, etc.

[0081] In this embodiment, the preset indicators include the score call volume, score average, score scorability rate, score hit rate and score abnormal request rate calculated by minute, hour and day.

[0082] In this embodiment, the month-on-month increase or decrease refers to the increase or decrease in the corresponding value of the preset indicator in the previous and subsequent n periods.

[0083] In this embodiment, the absolute value change rate is the change rate of the absolute value of the value corresponding to the preset indicator within one period.

[0084] In this embodiment, an early warning rule is designed from a preset dimension based on the data source score, and based on the early warning rule, the month-on-month increase or decrease and the rate of change of the absolute value of the preset indicator are determined. For example, technical indicators such as the score call volume, score average, score scorable rate, score hit rate and score abnormal request rate are calculated by minutes, hours and days, and the month-on-month increase or decrease and the rate of change of the absolute value of the technical indicators are monitored from dimensions such as customer organization unit, product service code unit, product version unit, and third-party data platform unit.

[0085] In this embodiment, the optimization of the warning rule may be performed by, for example, using parameters such as the triggering times, accuracy, and misjudgment rate of the warning rule as optimization indicators.

[0086] The beneficial effects of the above design scheme are: by selecting preset dimensions and preset indicators to determine the early warning rules, ensure the comprehensiveness of the establishment of early warning rules, provide a basis for the evaluation of third-party data, and conduct abnormal analysis of third-party data according to the early warning rules and issue early warning push, so as to facilitate efficient and timely early warning of faults, quickly troubleshoot problems, quickly make response plans, and improve operational efficiency. Finally, analyze the early warning results under the early warning rules, optimize the early warning rules, and improve the accuracy of the early warning rules.

[0087] Example 2

[0088] Based on Example 1, the embodiment of the present invention provides a real-time monitoring and early warning method for a third-party data information platform. In step 1, collecting data source scores corresponding to different customer organizations, scoring products, model versions, and time point information of the third-party data platform includes:

[0089] Obtain scoring rules for different client organizations, scoring products, model versions, and time point information on third-party data platforms;

[0090] Based on the scoring rules, the data sources corresponding to the different client organizations, scoring products, model versions, and time point information are scored to obtain data source scores.

[0091] The beneficial effect of the above design scheme is: by using the third-party data platform to score according to the scoring rules of different client institutions, scoring products, model versions, and time point information, a basis is provided for early warning analysis of third-party data.

[0092] Example 3

[0093] Based on Example 2, the embodiment of the present invention provides a real-time monitoring and early warning method for a third-party data information platform, which, after scoring the data sources corresponding to the different client organizations, scoring products, model versions, and time point information, further includes:

[0094] Perform anomaly detection on the scored data sources and extract the target data sources with missing scores;

[0095] According to the mapping relationship, the target data source is filled and processed to obtain a score of the target data source.

[0096] In this embodiment, since there are a lot of special cases other than normal scoring in the 2 data source scoring, such as scoring timeout, missed customer groups, etc., all special cases need to be filled with specified values.

[0097] In this embodiment, the mapping relationship is preset according to a scoring rule.

[0098] The beneficial effect of the above design scheme is that by filling in the target data sources with missing scores, the comprehensiveness and rationality of the data source scores are guaranteed.

[0099] Example 4

[0100] Based on Example 1, the present invention provides a real-time monitoring and early warning method for a third-party data information platform, such as Figure 2 As shown, in step 2, based on the historical data source score, the month-on-month increase or decrease and the absolute value change rate of each preset indicator under the preset dimension are determined, including:

[0101] Step 2-1: Based on the preset dimension, the historical data source scores are divided to obtain the relevant data source scores corresponding to each preset dimension;

[0102] Step 2-2: Under the preset dimension, the scores of the relevant data sources are analyzed according to the preset indicators to determine the month-on-month increase or decrease and the rate of change of the absolute value under each preset indicator.

[0103] In this embodiment, the preset dimensions include, for example, customer organization units, product service code units, product version units, third-party data platform units, etc., and data source scores related to the preset dimensions are obtained to facilitate analysis of preset indicators under the dimensions.

[0104] The beneficial effect of the above design scheme is: by selecting preset dimensions and preset indicators to analyze the data source scores, the month-on-month increase or decrease and the rate of change of the absolute value are calculated, providing a basis for the establishment of early warning rules.

[0105] Example 5

[0106] Based on Example 4, the embodiment of the present invention provides a real-time monitoring and early warning method for a third-party data information platform, analyzing the scores of the relevant data sources according to the preset indicators, and determining the month-on-month increase or decrease and the absolute value change rate under each preset indicator, including:

[0107] The preset indicators include the rating call volume, the average rating, the rating score availability rate and the rating abnormality request rate;

[0108] The scoring call volume is divided into call volume and effective call volume. The call volume is determined by the sum of all calls of each client institution to different scoring products and model versions according to the relevant data source scores; the effective call volume is the sum of normal calls determined by making special calls;

[0109] The score is divided into the average of the scores of the normal data sources in the scores of the relevant data sources;

[0110] The scoring rate is the ratio between the effective call volume and the call volume;

[0111] The scoring abnormal request rate is the ratio between the number of abnormal calls with invalid scoring and scoring timeout and the number of calls;

[0112] Set a grading period and set the interval time under the grading period;

[0113] Based on the scoring cycle and interval time, the month-on-month increase or decrease and the absolute value change rate of each preset indicator are calculated.

[0114] The beneficial effects of the above design scheme are: by clarifying the calculation method of each preset indicator to obtain the values ​​of all preset indicators under each preset dimension, the preset indicators can accurately represent the data source score and provide a basis for the establishment of early warning rules.

[0115] Example 6

[0116] Based on Example 1, the embodiment of the present invention provides a real-time monitoring and early warning method for a third-party data information platform. In step 2, based on the month-on-month increase or decrease and the rate of change of the absolute value, an early warning rule is established, including:

[0117] Obtaining warning information from historical data sources, and determining the reference month-on-month increase / decrease and reference change rate for preliminarily determining that data anomalies occur based on the warning information from the month-on-month increase / decrease and the change rate of the absolute value;

[0118] Based on the characteristics of the preset dimension, determining the importance coefficients of different preset indicators to the preset dimension;

[0119] Based on the historical monitoring information of the preset dimension, determining a first warning threshold value of each preset indicator under the preset dimension;

[0120] Based on the importance coefficient, the first warning threshold is corrected to obtain a second warning threshold;

[0121] Based on the characteristics of the preset dimensions, determining the correlation characteristics between the preset dimensions, and based on the correlation characteristics, determining the correlation relationship between different preset dimensions and the same preset indicator;

[0122] Based on the association relationship, determine the value constraint relationship between the same preset indicator under different preset dimensions;

[0123] Based on the value constraint relationship, determining the value difference range of the different preset dimensions between the second warning thresholds of the same preset indicator;

[0124] Extracting the second warning threshold to be adjusted whose second warning threshold does not meet the value difference range, and adjusting the second warning threshold to be adjusted based on the correlation relationship between the preset indicators corresponding to the second warning threshold to be adjusted and the value difference range;

[0125] Obtain the preset month-on-month increase or decrease and the preset change rate determined by the adjusted second warning threshold;

[0126] Determine whether the difference between the preset month-on-month increase / decrease and the preset change rate and the reference month-on-month increase / decrease and the reference change rate is within a preset range;

[0127] If yes, the preset month-on-month increase / decrease and the preset change rate are used as threshold values ​​of the warning rule to establish the warning rule;

[0128] Otherwise, a new value difference range is re-determined, and the second warning threshold to be adjusted is adjusted until the preset month-on-month increase or decrease and the preset change rate determined by the adjusted second warning threshold meet the requirements.

[0129] In this embodiment, the importance coefficient is used to indicate the importance of the preset indicator to the preset dimension. For example, when the preset dimension is the client organization unit, the preset indicator is the call volume which can better reflect the characteristics of the client organization unit, and the importance coefficient corresponding to the call volume is larger.

[0130] In this embodiment, the first warning threshold is obtained by weighting the first warning threshold according to the importance coefficient, for example, to obtain the second warning threshold.

[0131] In this embodiment, the reference month-on-month increase / decrease and the reference change rate are determined based on historical results.

[0132] In this embodiment, the features of the preset dimensions are used to represent the attributes of the preset dimensions, such as client organizations, rated products, etc.

[0133] In this embodiment, the value constraint relationship is, for example, that the difference between the average value of the preset indicator score under the preset dimension of client organization and the average value of the preset indicator score under the preset dimension of engraving score does not exceed the preset value.

[0134] In this embodiment, the second warning threshold to be adjusted is adjusted to ensure that the adjusted second warning threshold can more accurately reflect the characteristics of the data source score, thereby ensuring the accuracy of establishing the warning rule.

[0135] The beneficial effects of the above design scheme are: determining the warning threshold according to the correlation between the preset dimensions, and judging and adjusting the warning threshold by referring to the reference month-on-month increase / decrease and reference change rate determined by the historical data source, so that the month-on-month increase / decrease and the preset change rate finally determined according to the warning threshold can better represent the abnormal judgment of the data source, thereby ensuring the accuracy of the warning rules.

[0136] Example 7

[0137] Based on Example 1, the embodiment of the present invention provides a real-time monitoring and early warning method for a third-party data information platform. In step 3, according to the early warning rule, it is determined whether the data source score has data anomalies, including:

[0138] Based on the warning rules, determine the month-on-month increase / decrease threshold and the absolute value change rate threshold of the preset indicator under each preset dimension;

[0139] Based on the month-on-month increase / decrease threshold and the absolute value change rate threshold, the monitoring month-on-month increase / decrease and the monitoring absolute value change rate are graded, and the warning sub-content corresponding to the preset indicator under each preset dimension under each grade is determined;

[0140] Establish a data warning sub-content table based on the month-on-month increase / decrease threshold, the absolute value change rate threshold, and the warning sub-content;

[0141] Performing a correlation analysis on any two warning sub-contents to determine whether there is a correlation between the two warning sub-contents; if there is a correlation, performing a combined analysis on the two warning sub-contents to determine the relationship type corresponding to the two warning sub-contents;

[0142] Based on the relationship type, merging the two warning sub-contents to obtain warning content;

[0143] Based on the warning content, the data warning sub-content table is updated to obtain a warning content table;

[0144] Obtain the actual month-on-month increase or decrease and actual change rate of the data source score under each preset dimension and preset indicator;

[0145] Matching the actual month-on-month increase / decrease and the actual change rate with the warning content table to determine whether the match is successful;

[0146] If so, determine that data anomalies occur in the data source score, and determine the actual warning content based on the matching result;

[0147] Otherwise, it is determined that no data anomaly occurs in the data source scoring.

[0148] In this embodiment, there are multiple thresholds for the rate of change of the month-on-month increase / decrease threshold and the absolute value. For example, the first warning sub-content corresponds to the value between the first month-on-month increase / decrease threshold and the absolute value and the second month-on-month increase / decrease threshold and the absolute value, the second warning sub-content corresponds to the value less than the first month-on-month increase / decrease threshold and the absolute value, and the third warning sub-content corresponds to the value greater than the second month-on-month increase / decrease threshold and the absolute value.

[0149] In this embodiment, there may be correlation between the values ​​of the preset indicators in different preset dimensions, so it is necessary to perform correlation analysis on any two warning sub-contents.

[0150] In this embodiment, the relationship type may be, for example, a superposition relationship, an exponential relationship, a cancellation relationship, etc.

[0151] In this embodiment, the warning content table is a warning content corresponding to a plurality of increase / decrease thresholds and absolute value ranges.

[0152] The beneficial effect of the above design scheme is: by analyzing the early warning rules, determining the month-on-month increase or decrease threshold and the absolute value change rate threshold, and the correlation between different preset dimensions, further determining the comprehensive early warning content, ensuring the comprehensiveness and accuracy of the early warning content, and improving the accuracy of early warning data anomalies.

[0153] Example 8

[0154] Based on Example 7, the embodiment of the present invention provides a real-time monitoring and early warning method for a third-party data information platform. In step 3, after data anomalies occur, early warning push is performed, including:

[0155] Determine the actual warning content according to the matching result, and generate an alarm email according to the actual warning content;

[0156] The alarm email is pushed to the mailbox of the monitoring personnel.

[0157] The beneficial effect of the above design is that emails are sent to monitoring personnel in a timely manner at the data airport, facilitating rapid response and troubleshooting of data problems.

[0158] Example 9

[0159] Based on Example 1, the embodiment of the present invention provides a real-time monitoring and early warning method for a third-party data information platform. In step 4, the early warning result under the early warning rule within a preset time is obtained, and the early warning result is analyzed. The early warning rule is optimized according to the analysis result, including:

[0160] Based on the warning results, determine the number of warnings within a preset time, and the data investigation results corresponding to each warning;

[0161] Determine the accurate value of each warning based on the warning results and data investigation results;

[0162] The calculation formula for the accurate value F of the current warning is as follows:

[0163]

[0164] Among them, P S The warning analysis value representing the warning result of the current warning is (0, 1). Y Indicates the investigation and analysis value based on the data investigation result of the current warning, and its value is (0, 1). S The abnormal data range value corresponding to the warning result of the current warning is (0, 1), α Y The abnormal data range value corresponding to the data screening result of the current warning is (0, 1);

[0165] Based on the number of warnings and the accuracy value, an analysis result is obtained, and a warning result in the analysis result in which the accuracy value is greater than a preset accuracy value is taken as a positive sample, and other warning results are taken as negative samples;

[0166] Determining the accuracy of the warning rule based on the positive samples and the negative samples;

[0167] The calculation formula of the accuracy rate G of the early warning rule is as follows:

[0168]

[0169] Among them, K a Represents the first weight value, K b represents the second weight value, n represents the number of positive samples, m represents the number of negative samples, F ai represents the exact value corresponding to the i-th positive sample, F bj Represents the exact value corresponding to the jth negative sample;

[0170] Using the warning result corresponding to the positive sample as an iterative rule group, and determining the number of iterations according to the accuracy rate;

[0171] Based on the iterative rule group, the early warning rule is iteratively optimized according to the number of iterations to obtain an optimized early warning rule.

[0172] In this embodiment, the values ​​of the first weight value and the second weight value are both (0, 1), which is determined by the value of the preset accurate value. The larger the preset accurate value, the larger the corresponding K a The smaller the K b The larger the K a Greater than K b .

[0173] In this embodiment, the higher the accuracy, the smaller the corresponding number of iterations.

[0174] In this embodiment, the accuracy value indicates the accuracy of an early warning result.

[0175] In this embodiment, the accuracy rate represents the overall accuracy rate of all warning results determined according to the warning rules within a preset time.

[0176] In this embodiment, the larger the abnormal data range is, the larger the value is.

[0177] The beneficial effect of the above design scheme is: by obtaining the warning results under the warning rules within a preset time, and comparing and analyzing the warning results and the actual data investigation results, and optimizing the warning rules according to the analysis results, the accuracy of the warning rules is improved.

[0178] Example 10

[0179] A real-time monitoring and early warning system for a three-party data information platform, such as Figure 3 As shown, including:

[0180] The data source module is used to collect data source scores corresponding to different client organizations, scoring products, model versions, and time point information from third-party data platforms;

[0181] A rule design module is used to determine the month-on-month increase or decrease and the absolute value change rate of each preset indicator under the preset dimension based on the historical data source score, and establish an early warning rule based on the month-on-month increase or decrease and the absolute value change rate;

[0182] An early warning push module is used to determine whether data anomalies occur in the data source score according to the early warning rule, and push an early warning if data anomalies occur;

[0183] The rule optimization module is used to obtain the warning results under the warning rules within a preset time, analyze the warning results, and optimize the warning rules according to the analysis results.

[0184] The beneficial effects of the above design scheme are: by selecting preset dimensions and preset indicators to determine the early warning rules, ensure the comprehensiveness of the establishment of early warning rules, provide a basis for the evaluation of third-party data, and conduct abnormal analysis of third-party data according to the early warning rules and issue early warning push, so as to facilitate efficient and timely early warning of faults, quickly troubleshoot problems, quickly make response plans, and improve operational efficiency. Finally, analyze the early warning results under the early warning rules, optimize the early warning rules, and improve the accuracy of the early warning rules.

[0185] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A real-time monitoring and early warning method for a third-party data information platform, characterized in that: include: Step 1: Collect data source scores from third-party data platforms for different client organizations, scoring products, model versions, and time point information; Step 2: Based on the historical data source scores, determine the month-on-month increase or decrease and the absolute value change rate of each preset indicator under the preset dimension, and establish an early warning rule based on the month-on-month increase or decrease and the absolute value change rate; Step 3: According to the warning rule, determine whether the data source score has data anomalies, and push warnings when data anomalies occur; Step 4: Obtain the warning results under the warning rules within a preset time, analyze the warning results, and optimize the warning rules according to the analysis results; In step 2, based on the month-on-month increase or decrease and the absolute value change rate, an early warning rule is established, including: Obtaining warning information from historical data sources, and determining the reference month-on-month increase / decrease and reference change rate for preliminarily determining that data anomalies occur based on the warning information from the month-on-month increase / decrease and the change rate of the absolute value; Based on the characteristics of the preset dimension, determining the importance coefficients of different preset indicators to the preset dimension, the importance coefficients being used to indicate the importance of the preset indicator to the preset dimension; Based on the historical monitoring information of the preset dimension, determining a first warning threshold value of each of the preset indicators under the preset dimension; Based on the importance coefficient, the first warning threshold is corrected to obtain a second warning threshold; Based on the characteristics of the preset dimensions, determining the correlation characteristics between the preset dimensions, and based on the correlation characteristics, determining the correlation relationship between different preset dimensions and the same preset indicator; Based on the association relationship, determine the value constraint relationship between the same preset indicator under different preset dimensions; Based on the value constraint relationship, determining the value difference range of the different preset dimensions between the second warning thresholds of the same preset indicator; Extracting the second warning threshold to be adjusted whose second warning threshold does not meet the value difference range, and adjusting the second warning threshold to be adjusted based on the correlation relationship between the preset indicators corresponding to the second warning threshold to be adjusted and the value difference range; Obtain the preset month-on-month increase or decrease and the preset change rate determined by the adjusted second warning threshold; Determine whether the difference between the preset month-on-month increase / decrease and the preset change rate and the reference month-on-month increase / decrease and the reference change rate is within a preset range; If yes, the preset month-on-month increase / decrease and the preset change rate are used as threshold values ​​of the warning rule to establish the warning rule; Otherwise, a new value difference range is re-determined, and the second warning threshold to be adjusted is adjusted until the preset month-on-month increase or decrease and the preset change rate determined by the adjusted second warning threshold meet the requirements.

2. The real-time monitoring and early warning method of a third-party data information platform according to claim 1 is characterized in that: In step 1, the data source scores corresponding to different client organizations, scoring products, model versions, and time point information collected from the third-party data platform include: Obtain scoring rules for different client organizations, scoring products, model versions, and time point information on third-party data platforms; Based on the scoring rules, the data sources corresponding to the different client organizations, scoring products, model versions, and time point information are scored to obtain data source scores.

3. The real-time monitoring and early warning method of a third-party data information platform according to claim 2 is characterized in that: After scoring the data sources corresponding to the different client organizations, scoring products, model versions, and time point information, it also includes: Perform anomaly detection on the scored data sources and extract the target data sources with missing scores; According to the mapping relationship, the target data source is filled and processed to obtain a score of the target data source.

4. The real-time monitoring and early warning method of a third-party data information platform according to claim 1 is characterized in that: In step 2, based on the historical data source scores, the month-on-month increase or decrease and the absolute value change rate of each preset indicator under the preset dimension are determined, including: Based on the preset dimensions, the historical data source scores are divided to obtain relevant data source scores corresponding to each preset dimension; Under the preset dimension, the scores of the relevant data sources are analyzed according to the preset indicators to determine the month-on-month increase or decrease and the rate of change of the absolute value under each preset indicator.

5. The real-time monitoring and early warning method of a third-party data information platform according to claim 4 is characterized in that: The scores of the relevant data sources are analyzed according to the preset indicators to determine the month-on-month increase or decrease and the absolute value change rate under each preset indicator, including: The preset indicators include the rating call volume, the average rating, the rating score availability rate and the rating abnormality request rate; The scoring call volume is divided into call volume and effective call volume. The call volume is determined by the sum of all calls of each client institution to different scoring products and model versions according to the relevant data source scores; the effective call volume is the sum of normal calls determined by making special calls; The score is divided into the average of the scores of the normal data sources in the scores of the relevant data sources; The scoring rate is the ratio between the effective call volume and the call volume; The scoring abnormal request rate is the ratio between the number of abnormal calls with invalid scoring and scoring timeout and the number of calls; Set a grading period and set the interval time under the grading period; Based on the scoring cycle and interval time, the month-on-month increase or decrease and the absolute value change rate of each preset indicator are calculated.

6. The real-time monitoring and early warning method of a third-party data information platform according to claim 1 is characterized in that: In step 3, according to the warning rule, determining whether the data source score has data anomalies includes: Based on the warning rules, determine the month-on-month increase / decrease threshold and the absolute value change rate threshold of the preset indicator under each preset dimension; Based on the month-on-month increase / decrease threshold and the absolute value change rate threshold, the monitored month-on-month increase / decrease and the absolute value change rate are graded, and the early warning sub-content corresponding to the preset indicator under each preset dimension is determined at each grade; Establish a data warning sub-content table based on the month-on-month increase / decrease threshold, the absolute value change rate threshold, and the warning sub-content; Performing a correlation analysis on any two warning sub-contents to determine whether there is a correlation between the two warning sub-contents; if there is a correlation, performing a combined analysis on the two warning sub-contents to determine the relationship type corresponding to the two warning sub-contents; Based on the relationship type, merging the two warning sub-contents to obtain warning content; Based on the warning content, the data warning sub-content table is updated to obtain a warning content table; Obtain the actual month-on-month increase or decrease and actual change rate of the data source score under each preset dimension and preset indicator; Matching the actual month-on-month increase / decrease and the actual change rate with the warning content table to determine whether the match is successful; If so, determine that data anomalies occur in the data source score, and determine the actual warning content based on the matching result; Otherwise, it is determined that no data anomaly occurs in the data source scoring.

7. The real-time monitoring and early warning method of a third-party data information platform according to claim 6 is characterized in that: In step 3, after data anomalies occur, an early warning is pushed, including: Determine the actual warning content according to the matching result, and generate an alarm email according to the actual warning content; The alarm email is pushed to the mailbox of the monitoring personnel.

8. The real-time monitoring and early warning method of a third-party data information platform according to claim 1 is characterized in that: In step 4, the warning result under the warning rule within a preset time is obtained, and the warning result is analyzed, and the warning rule is optimized according to the analysis result, including: Based on the warning results, determine the number of warnings within a preset time, and the data investigation results corresponding to each warning; Determine the accurate value of each warning based on the warning results and data investigation results; Based on the number of warnings and the accuracy value, an analysis result is obtained, and a warning result in the analysis result in which the accuracy value is greater than a preset accuracy value is taken as a positive sample, and other warning results are taken as negative samples; Determining the accuracy of the warning rule based on the positive samples and the negative samples; Using the warning result corresponding to the positive sample as an iterative rule group, and determining the number of iterations according to the accuracy rate; Based on the iterative rule group, the early warning rule is iteratively optimized according to the number of iterations to obtain an optimized early warning rule.

9. A real-time monitoring and early warning system for a third-party data information platform, characterized in that: include: The data source module is used to collect data source scores corresponding to different client organizations, scoring products, model versions, and time point information from third-party data platforms; The rule design module is used to determine the month-on-month increase or decrease and the absolute value change rate of each preset indicator under the preset dimension based on the historical data source score, and establish early warning rules based on the month-on-month increase or decrease and the absolute value change rate, including: Obtaining warning information from historical data sources, and determining the reference month-on-month increase / decrease and reference change rate for preliminarily determining that data anomalies occur based on the warning information from the month-on-month increase / decrease and the change rate of the absolute value; Based on the characteristics of the preset dimension, determining the importance coefficients of different preset indicators to the preset dimension, the importance coefficients being used to indicate the importance of the preset indicator to the preset dimension; Based on the historical monitoring information of the preset dimension, determining a first warning threshold value of each of the preset indicators under the preset dimension; Based on the importance coefficient, the first warning threshold is corrected to obtain a second warning threshold; Based on the characteristics of the preset dimensions, determining the correlation characteristics between the preset dimensions, and based on the correlation characteristics, determining the correlation relationship between different preset dimensions and the same preset indicator; Based on the association relationship, determine the value constraint relationship between the same preset indicator under different preset dimensions; Based on the value constraint relationship, determining the value difference range of the different preset dimensions between the second warning thresholds of the same preset indicator; Extracting the second warning threshold to be adjusted whose second warning threshold does not meet the value difference range, and adjusting the second warning threshold to be adjusted based on the correlation relationship between the preset indicators corresponding to the second warning threshold to be adjusted and the value difference range; Obtain the preset month-on-month increase or decrease and the preset change rate determined by the adjusted second warning threshold; Determine whether the difference between the preset month-on-month increase / decrease and the preset change rate and the reference month-on-month increase / decrease and the reference change rate is within a preset range; If yes, the preset month-on-month increase / decrease and the preset change rate are used as threshold values ​​of the warning rule to establish the warning rule; Otherwise, a new value difference range is re-determined, and the second warning threshold to be adjusted is adjusted until the preset month-on-month increase or decrease and the preset change rate determined by the adjusted second warning threshold meet the requirements; An early warning push module is used to determine whether data anomalies occur in the data source score according to the early warning rule, and push an early warning if data anomalies occur; The rule optimization module is used to obtain the warning results under the warning rules within a preset time, analyze the warning results, and optimize the warning rules according to the analysis results.

Images