Data volume anomaly early warning method, device, equipment, storage medium and product

Abstract

The application discloses a data volume anomaly early warning method and device, equipment, a storage medium and a product, and relates to the technical field of anomaly early warning. The data volume anomaly early warning method comprises the following steps: acquiring a data volume collection table of the day; performing anomaly analysis on data volume in the data volume collection table by using a preset collection rule and a preset fluctuation rate feature to obtain an anomaly analysis result; and generating early warning information based on the anomaly analysis result. The application performs data volume anomaly analysis on the collection rule and the fluctuation rate feature of data in the data volume collection table to obtain an anomaly analysis result, and performs multi-angle anomaly monitoring, so that the accuracy of anomaly early warning is improved.

Description

Technical Field

[0001] This application relates to the field of anomaly warning technology, and in particular to a method, apparatus, device, storage medium and product for anomaly warning of data volume. Background Technology

[0002] Financial institutions are required to regularly collect key data such as their own operational data, financial data, and risk management information in accordance with laws, regulations, and regulatory requirements, and to accurately and completely report them to the relevant national financial regulatory agencies. This is to meet the supervision and management needs of financial regulatory agencies regarding the market operation, risk status, and compliance of financial institutions. Therefore, ensuring that the data is accurately reported to the relevant financial regulatory agencies is a crucial step in the data reporting process.

[0003] In related technologies, the main approach is to compare the reported result data with the data volume of historical reporting periods. By simply comparing the fluctuations or increases / decreases in the data volume of multiple periods, it is determined whether the collected data volume is normal. This analysis method relies on a single human experience judgment, which reduces the accuracy of the data volume anomaly warning. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, device, storage medium, and product for early warning of abnormal data volume, aiming to solve the technical problem of reduced accuracy of early warning of abnormal data volume.

[0005] To achieve the above objectives, this application proposes a data volume anomaly early warning method, which includes:

[0006] Obtain the data collection table for the current day;

[0007] Anomaly analysis is performed on the data volume in the data volume collection table using preset collection patterns and preset volatility characteristics to obtain anomaly analysis results;

[0008] Based on the anomaly analysis results, an early warning message is generated.

[0009] In one embodiment, the step of performing anomaly analysis on the data volume in the data volume collection table based on preset collection rules and preset volatility characteristics to obtain anomaly analysis results includes:

[0010] Obtain the historical sample data collection table;

[0011] Based on the historical sample data collection table, the data collection table for the current day is subjected to data collection pattern detection to obtain the first anomaly analysis result;

[0012] Based on the historical sample data collection table, fluctuation characteristics are detected on the data collection table for the current day to obtain the second anomaly analysis result;

[0013] The first anomaly analysis result and the second anomaly analysis result are used as the anomaly analysis results.

[0014] In one embodiment, the historical sample data collection table includes a previous historical sample data collection table. The step of performing data collection pattern detection on the current day's data collection table based on the historical sample data collection table to obtain a first anomaly analysis result includes:

[0015] Obtain the first data volume from the data volume collection table for the current day and the second data volume from the previous historical sample data volume collection table;

[0016] If the data volume collection table for the current day and the historical sample data volume collection table for the previous period belong to the same collection period, and the second data volume is greater than the first data volume, then an anomaly is determined, and the first anomaly analysis result is obtained.

[0017] In one embodiment, the historical sample data includes a historical sample data collection table within a preset time period. Before the step of performing fluctuation characteristic detection on the data collection table of the current day based on the historical sample data collection table to obtain the second anomaly analysis result, the following steps are included:

[0018] The sample volatility corresponding to the historical sample data collection table for each collection cycle within the preset time period is calculated.

[0019] The correlation of the sample volatility is calculated to obtain a correlation value. If the correlation value is greater than a preset correlation threshold, it is determined that the volatility of the historical sample data volume collection table for each collection period within the preset time period has a periodicity, so as to perform anomaly detection based on the periodicity of the volatility and obtain a second anomaly analysis result.

[0020] In one embodiment, the step of performing volatility characteristic detection on the data collection table of the current day based on the historical sample data collection table to obtain a second anomaly analysis result includes:

[0021] The first data volume is compared with the second data volume to obtain the target data volume volatility.

[0022] The maximum value of the historical sample data collection table within a preset time period is calculated according to the preset date calculation rules, so as to obtain the maximum value of the historical data volume and the maximum value of the historical volatility of the historical sample data collection table within the preset time period.

[0023] The maximum value of the historical data volume, the maximum value of the historical volatility, the target data volume volatility, and the first data volume are used as detection information;

[0024] The detection information is judged. If the volatility of the target data volume is less than or equal to a preset volatility threshold, the volatility of the target data volume is greater than or equal to the maximum value of the historical volatility by a preset multiple, and the first data volume is greater than the maximum value of the historical data volume, then an anomaly is determined, and a second anomaly analysis result is obtained.

[0025] In one embodiment, after the step of generating early warning information based on the anomaly analysis results, the method includes:

[0026] Obtain the initial early warning dataset;

[0027] The warning information is input into the initial warning dataset to obtain the target warning dataset;

[0028] Obtain the initial feedback dataset;

[0029] The initial feedback dataset is defined based on the warning information to obtain the target feedback dataset;

[0030] Establish a relationship between the target early warning dataset and the target feedback dataset to obtain a preset association relationship;

[0031] The target early warning dataset is sent to the user terminal according to the preset association relationship, so that the user terminal can input feedback information into the target feedback dataset for feedback based on the target early warning dataset.

[0032] Furthermore, to achieve the above objectives, this application also proposes a data volume anomaly early warning device, which includes:

[0033] The acquisition module is used to obtain the data collection table for the current day;

[0034] The analysis module is used to perform anomaly analysis on the data volume in the data volume collection table based on preset collection rules and preset volatility characteristics, and to obtain anomaly analysis results.

[0035] The early warning module is used to generate early warning information based on the anomaly analysis results.

[0036] In addition, to achieve the above objectives, this application also proposes a data volume anomaly early warning device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the data volume anomaly early warning method as described above.

[0037] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the data volume anomaly early warning method described above.

[0038] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the data volume anomaly warning method described above.

[0039] One or more technical solutions proposed in this application have at least the following technical effects:

[0040] This application automatically performs anomaly analysis on the data collection patterns and volatility characteristics of the data in the data collection table, obtains anomaly analysis results, and improves the accuracy of anomaly early warning through multi-angle anomaly monitoring based on collection patterns and volatility characteristics. Attached Figure Description

[0041] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating an embodiment of the data volume anomaly early warning method of this application.

[0044] Figure 2 This is a framework diagram for the data volume anomaly early warning method of this application;

[0045] Figure 3 This is a flowchart illustrating the anomaly analysis process of the data volume anomaly early warning method in this application.

[0046] Figure 4 The flowchart shows the implementation of the abnormal data collection pattern detection method in the data volume abnormality early warning method of this application.

[0047] Figure 5 This is an abnormal case diagram illustrating the abnormal data collection pattern detection of the data volume abnormality early warning method in this application.

[0048] Figure 6 The flowchart shows the implementation process of incremental table anomaly analysis in the data volume anomaly early warning method of this application.

[0049] Figure 7 The flowchart shows the full-scale table anomaly analysis implementation of the data volume anomaly early warning method in this application.

[0050] Figure 8 This is a full-scale table anomaly case diagram for the data volume anomaly early warning method in this application;

[0051] Figure 9 This is a flowchart illustrating Embodiment 2 of the data volume anomaly early warning method of this application;

[0052] Figure 10 This is a schematic diagram illustrating the periodicity of data volume fluctuations in the data volume anomaly early warning method of this application;

[0053] Figure 11 The flowchart shows the implementation of the volatility characteristic detection method for the data volume anomaly early warning method in this application.

[0054] Figure 12 This is a case study illustrating the fluctuation characteristics of the data volume anomaly early warning method in this application.

[0055] Figure 13 A flowchart of data processing visualization for the abnormal data volume abnormality early warning report of the data volume abnormality early warning method in this application;

[0056] Figure 14 This is a visual view of the early warning report for the data volume anomaly early warning method in this application;

[0057] Figure 15 This is a screenshot of the notification function interface for the data volume anomaly warning method in this application;

[0058] Figure 16 This is a schematic diagram of the module structure of the data volume abnormality early warning device according to an embodiment of this application;

[0059] Figure 17 This is a schematic diagram of the device structure of the hardware operating environment involved in the data volume anomaly early warning method in this application embodiment. Detailed Implementation

[0060] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0061] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0062] The main solution of this application embodiment is: to obtain the data volume collection table for the day; to perform anomaly analysis on the data volume in the data volume collection table through preset collection rules and preset volatility characteristics, and to obtain anomaly analysis results; and to generate early warning information based on the anomaly analysis results.

[0063] In related technologies, the main approach is to compare the reported result data with the data volume of historical reporting periods. By simply comparing the fluctuations or increases / decreases in the data volume of multiple periods, it is determined whether the collected data volume is normal. This analysis method relies on a single human experience judgment, which reduces the accuracy of the data volume anomaly warning.

[0064] This application automatically performs anomaly analysis on the data collection patterns and volatility characteristics of the data in the data collection table, obtains anomaly analysis results, and improves the accuracy of anomaly early warning through multi-angle anomaly monitoring based on collection patterns and volatility characteristics.

[0065] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions. The following description uses a data volume anomaly warning device as an example to illustrate this embodiment and the subsequent embodiments.

[0066] Based on this, embodiments of this application provide a method for early warning of abnormal data volume, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the data volume anomaly warning method of this application.

[0067] In this embodiment, refer to Figure 2 , Figure 2 This is a framework diagram for data volume anomaly early warning. The data volume anomaly early warning method includes steps S100 to S300:

[0068] Step S100: Obtain the data collection table for the day;

[0069] It should be noted that the execution entity in this embodiment is a data volume anomaly early warning device. This device is equipped with a data warehouse. Therefore, the anomaly early warning device can obtain the daily data volume collection table from the data warehouse. This daily data volume collection table is generated by the data warehouse through a data volume statistics program, which calculates the data volume of data blocks by table. The data volume in this table represents the data volume of the previous day. The detailed table structure of the daily data volume collection table obtained by the anomaly early warning device is shown in Table 1.

[0070] Among them, "Data Date (DW_DAT_DT)" indicates the data collection date of the corresponding report. For example, data collected on April 1, 2023, will be dated March 31, 2023. "Table Chinese Name (TBL_CH_NM)" and "Table English Name (TBL_EN_NM)" are the table names defined in the document model, such as "Employee Table" and "YGB". "Data Block ID (BLK_ID)" is a label that distinguishes the data range in the table. It is mainly a data ownership label formulated to implement the regulatory unit's requirement to "clarify the responsibility of data quality issues between data departments and business departments". The data range is divided into different responsible persons according to the data block ID label, which facilitates data traceability and problem follow-up. The "Data Block ID" is also used to identify abnormal issues and follow up with users. "Number of Records (NUM)" is the total number of records corresponding to the data block counted by the data volume statistics program. "Statistical Time (STAT_TM)" is the time point at which the total number of records of the data block was counted.

[0071] Table 1. Statistical Results and Sample Data

[0072] ...... 2024-03-31 YGB 0102_RLZY_SAP Employee List 10543 2024-04-0116:08:32 2024-04-01 YGB 0102_RLZY_SAP Employee List 16679 2024-04-0211:25:07 2024-04-02 YGB 0102_RLZY_SAP Employee List 16680 2024-04-0313:46:15 ....

[0073] Step S200: Perform anomaly analysis on the data volume in the data volume collection table using preset collection rules and preset volatility characteristics to obtain anomaly analysis results;

[0074] Understandably, the preset data collection pattern is derived by analyzing the patterns in the daily data collection table. The preset fluctuation characteristic is the periodic characteristic of the data change rate. The data volume anomaly early warning device mainly calculates and judges whether the report data volume is abnormal based on the data collection pattern and the time series characteristics of the data volume fluctuation rate. The anomaly analysis results are then obtained through SQL processing within the data warehouse.

[0075] In one feasible implementation, refer to Figure 3 , Figure 3 This is a flowchart of the anomaly analysis process for the data volume anomaly early warning method of this application. Step S200 includes the following steps:

[0076] Step S210: Obtain the historical sample data collection table;

[0077] It should be noted that the historical sample data volume collection table refers to the historical sample data volume collection table used when performing data volume anomaly analysis on the data volume collection table for the current day. The data volume anomaly early warning device obtains the required historical sample data volume collection table.

[0078] Step S220: Based on the historical sample data collection table, perform data collection pattern detection on the data collection table for the current day to obtain the first anomaly analysis result;

[0079] It should be noted that the first anomaly analysis result is an anomaly detection result obtained based on a preset data collection pattern. The data volume anomaly early warning device performs data collection pattern detection on the daily data volume collection table based on the obtained comparison results to obtain the anomaly analysis result.

[0080] In a first feasible implementation, step S220 includes the following steps:

[0081] Obtain the first data volume from the data volume collection table for the current day and the second data volume from the previous historical sample data volume collection table;

[0082] Understandably, the data volume anomaly warning device acquires the first data volume from the daily data volume collection table and the second data volume from the previous historical sample data volume collection table.

[0083] If the data volume collection table for the current day and the historical sample data volume collection table for the previous period belong to the same collection period, and the second data volume is greater than the first data volume, then an anomaly is determined, and the first anomaly analysis result is obtained.

[0084] It should be noted that the data volume collection table for the same collection period is the data volume collection table within the collection period of a single data reporting. The data volume collection table includes a full table and an incremental table. The full table requires the reporting of data valid up to the collection date, as well as all data considered to be in a final state, such as data that has been settled, expired, or terminated from the previous collection date to the current collection date. The incremental table in the data volume collection table requires the collection of data newly added from the previous collection date to the current collection date each time. If the data collection table is collected daily and monthly, then for the incremental table, the daily incremental data collected covers new transaction data occurring from the 1st of the month to the collection date. Whenever a new transaction occurs, the transaction data volume in the incremental table will increase. Furthermore, for the full table, the collected data covers valid data up to the end of the month, as well as all data considered finalized, closed, expired, or terminated within the month. Therefore, valid and expired data must be reported from the beginning to the end of the month. Based on this, it can be understood that the reporting covers the entire business scope. Under daily collection, whenever a new business occurs, the report data volume will also increase. Based on the above patterns, it can be inferred that the daily data volume collected by the incremental and full tables from the 1st of the month to the end of the month will maintain a steady trend. If a downward trend appears, it is an anomaly warning that requires monitoring and analysis to avoid missed reports. Therefore, the data volume anomaly early warning device compares the data volume of the first data volume collection table belonging to the same collection period with the data volume of the previous historical sample data volume collection table. If the second data volume is greater than the first data volume, an anomaly is determined, and the first anomaly analysis result is obtained. Please refer to [the relevant documentation]. Figure 4 , Figure 4 A flowchart for detecting anomalies in data collection patterns is provided.

[0085] In practice, if the system batch processing fails on July 26th and data is not generated, please refer to [the relevant documentation / reference]. Figure 5 , Figure 5 Anomaly case diagrams for detecting abnormal patterns in data collection are provided.

[0086] In a second feasible implementation, step S220 includes the following steps:

[0087] If the current day and the previous day do not belong to the same collection period, an anomaly analysis is performed on the comparison result. If the data volume of the incremental table in the first data volume collection table is greater than the data volume of the historical incremental table in the historical sample data volume collection table of the previous day, an anomaly is determined, and an incremental table anomaly result is obtained. If the data volume of the full table in the first data volume collection table decreases by more than the data volume of the historical full table in the historical sample data volume collection table of the previous day by more than a preset full table decrease range, an anomaly is determined, and a full table anomaly result is obtained. Based on the incremental table anomaly result and the obtained full table anomaly result, a first anomaly analysis result is obtained.

[0088] Understandably, if the data collection table is collected daily and monthly, then for the incremental table, under the daily collection frequency, all new transactions for the current month need to be collected by the end of the month as required by regulations, and the collection of new transactions for the following month will restart on the 1st of the next month. Therefore, the data volume at the end of the previous month is generally larger than that at the beginning of the current month, because the former includes transaction data for the entire previous month, while the latter only includes transaction data for the first day of the current month. Therefore, it can be deduced that if the incremental table shows a larger data volume at the beginning of the month than at the end of the previous month when crossing months, it is an anomaly warning, requiring close monitoring and analysis to avoid over-reporting. Please refer to [reference needed]. Figure 6 , Figure 6 A flowchart for incremental table anomaly analysis is provided. For full tables, with daily data collection, there may be instances where expired business data from the previous month is not collected and reported when crossing months. Therefore, there is a possibility of a decrease in data volume when crossing months, but generally the decrease is small because existing business data will not be concentrated in a single month, causing a large number of expired data to be omitted from the following month. Therefore, it can be deduced that if a significant decrease in data volume occurs when the full table crosses months, it is an anomaly warning. In this case, close monitoring and analysis are necessary to avoid missed reports. Please refer to [the relevant documentation / reference]. Figure 7 , Figure 7 A flowchart for full-scale table anomaly analysis is provided. Therefore, the data volume anomaly early warning device judges the anomalies of the full-scale table and incremental table according to the above rules, and obtains the first anomaly analysis result based on the anomaly judgment result.

[0089] In practice, if adjustments to the data definition result in a significant decrease in the amount of data spanning April 1st compared to the end of the previous month, please refer to [the relevant guidelines]. Figure 8 , Figure 8 A diagram of anomaly cases from the full table is provided.

[0090] The above are only two feasible implementations of step S220 provided in this embodiment. This embodiment does not specifically limit the specific implementation of step S220.

[0091] Step S230: Based on the historical sample data collection table, perform fluctuation characteristic detection on the data collection table for the current day to obtain the second anomaly analysis result;

[0092] Understandably, volatility characteristics refer to the time-series characteristics of data volume fluctuations, reflecting the magnitude of quantity changes. If the data volume collection table is collected daily and monthly, the data volume anomaly early warning device compares the data volume and volatility in the current day's data volume collection table with the data volume and volatility in the data volume collection tables of the past three months and the same period on the current day. If it is significantly higher than the values ​​of the same period in the past three months, it is judged as an anomaly, and the second anomaly analysis result is obtained.

[0093] Step S240: The first anomaly analysis result and the second anomaly analysis result are used as the anomaly analysis result.

[0094] It should be noted that the data volume anomaly early warning device will use the first and second anomaly analysis results obtained through anomaly analysis as the anomaly analysis results.

[0095] Step S300: Based on the anomaly analysis results, generate early warning information.

[0096] Understandably, the warning information includes the verification date, warning category, warning sub-category, data block ID, reporting period, reporting contact person, and warning level information. The data volume anomaly warning device generates warning information based on the anomaly analysis results. This warning information includes the verification date, warning category, warning sub-category, data block ID, reporting period, reporting contact person, warning level information, and data volume.

[0097] This embodiment performs automatic anomaly analysis on the data volume collection table by using preset collection rules and preset volatility characteristics to obtain a first anomaly analysis result and a second anomaly analysis result. Based on the first and second anomaly results, it generates early warning information, thereby improving the efficiency of data volume anomaly early warning.

[0098] This application provides a method for early warning of abnormal data volume, referring to... Figure 9 , Figure 9This is a flowchart illustrating the second embodiment of the data volume anomaly warning method of this application.

[0099] In a first feasible implementation, steps A10 to A20 are included before step S230:

[0100] The sample volatility corresponding to the historical sample data collection table for each collection cycle within the preset time period is calculated.

[0101] It should be noted that the data volume anomaly early warning device calculates the sample volatility of the historical sample data volume collection table for each collection period.

[0102] The correlation of the sample volatility is calculated to obtain a correlation value. If the correlation value is greater than a preset correlation threshold, it is determined that the volatility of the historical sample data volume collection table for each collection period within the preset time period has a periodicity, so as to perform anomaly detection based on the periodicity of the volatility and obtain a second anomaly analysis result.

[0103] Understandably, correlation coefficients are indicators used to measure the strength and direction of the linear relationship between two variables. Data anomaly early warning devices use the Pearson correlation coefficient to analyze whether data volatility exhibits periodicity. The Pearson correlation coefficient is a statistical measure used to measure the linear correlation between two continuous variables. Calculating the Pearson correlation coefficient requires the covariance of the data and the standard deviations of the two variables. The detailed formula is as follows:

[0104] The Pearson correlation coefficient ranges from -1 to 1. When the value is closer to 1, it indicates a stronger positive correlation between the two variables, meaning that if one variable increases, the other variable also increases by the same proportion. When the value is closer to -1, it indicates a perfect negative correlation between the two variables, meaning that if one variable increases, the other variable decreases by the same proportion.

[0105] In addition, the data volume anomaly early warning device determines whether there is a monthly cycle in the data volume volatility of a data block by judging the mean Pearson correlation coefficient C between the data volume volatility of corresponding data blocks over three months.

[0106]

[0107] The volatility is calculated using the following formula, where RM-3, RM-2, and RM-1 represent the volatility of the corresponding data blocks for the previous three months.

[0108]

[0109] If the C value is greater than 0.6, it is determined that the volatility of the corresponding data block exhibits monthly periodicity, and this portion of the data block will be included in the monitoring scope. Please refer to [link / reference]. Figure 10 , Figure 10 A diagram illustrating the periodicity of data volume fluctuations is provided.

[0110] In a first feasible implementation, step S230 includes the following steps:

[0111] The first data volume is compared with the second data volume to obtain the target data volume volatility.

[0112] It should be noted that the target data volume volatility rate is the numerical change in the first data volume compared to the second data volume. The anomaly warning device compares the first and second data volumes to obtain the numerical change in the first data volume compared to the second data volume.

[0113] The maximum value of the historical sample data collection table within a preset time period is calculated according to the preset date calculation rules, so as to obtain the maximum value of the historical data volume and the maximum value of the historical volatility of the historical sample data collection table within the preset time period.

[0114] Understandably, if the data collection table collects data daily and monthly, then the current day is designated as day T. This means the data collected on that day is from day T-1. The preset date calculation rule is as follows: if date T = 2, 3, 4, or 5, then calculate the daily volatility and data volume for the past three months from the 2nd to the 8th, and find the maximum value. If date T >= 6, then calculate the daily volatility and data volume for the past three months from T-4 to T+3, and find the maximum value. Since changes in data volume have a relatively large impact on data volume fluctuations when the data volume is small, to reduce false positives, this method only performs the above-mentioned periodic fluctuation anomaly detection and early warning for data blocks corresponding to the historical three-month period with a maximum data volume greater than or equal to 100 and a first data volume greater than or equal to 1000. The data volume anomaly early warning device calculates the historical maximum data volume and historical maximum volatility based on the above rules.

[0115] The maximum value of the historical data volume, the maximum value of the historical volatility, the target data volume volatility, and the first data volume are used as detection information;

[0116] It should be noted that the data volume anomaly early warning device uses the calculated maximum historical data volume, the maximum historical volatility, the target data volume volatility, and the first data volume as detection information, so as to perform volatility characteristic detection based on the detection information later.

[0117] The detection information is judged. If the volatility of the target data volume is less than or equal to a preset volatility threshold, the volatility of the target data volume is greater than or equal to the maximum value of the historical volatility by a preset multiple, and the first data volume is greater than the maximum value of the historical data volume, then an anomaly is determined, and a second anomaly analysis result is obtained.

[0118] Understandably, the data volume anomaly early warning device judges the detected information according to preset judgment rules. Specifically, if the data volume collection table's collection frequency is daily and the collection cycle is monthly, the preset judgment rule is: if R... T-1 <=10% and R T-1 >=3R MAX And the first data size is NUM T-1 Greater than the historical data volume for the same period in March (NUM) MAX If the result is positive, it is detected as an anomaly, and the second anomaly analysis result is obtained. Where R... T-1 NUM represents the volatility of the target data volume from day T-1 to day T-2 in the current month. T-1 R represents the data volume for day T-1 of the current month; MAX NUM represents the highest daily volatility over the same period in the past three months. MAX This represents the maximum data volume for the same period over the past three months.

[0119] In a second feasible implementation, step S230 further includes the following steps:

[0120] The detection information is judged. If the volatility of the target data volume is within the preset volatility threshold, the volatility of the target data volume is greater than or equal to the maximum historical volatility value by a preset multiple, and the first data volume is greater than the historical maximum value, then an anomaly is determined, and a second anomaly analysis result is obtained.

[0121] It should be noted that the data volume anomaly early warning device judges the detected information according to preset judgment rules. Specifically, if the data volume collection table collects data daily and monthly, the preset judgment rule is: if 10%... <R T-1 <= 50% and R T-1 >=2R MAX And the first data size is NUM T-1 Greater than the historical data volume for the same period in March (NUM) MAX If the result is positive, it is detected as an anomaly, and the second anomaly analysis result is obtained.

[0122] In a third feasible implementation, step S230 further includes the following steps:

[0123] The detection information is judged. If the volatility of the target data volume is greater than a preset volatility threshold, the volatility of the target data volume is greater than or equal to the maximum historical volatility value by a preset multiple, and the first data volume is greater than the historical maximum value, then an anomaly is determined, and a second anomaly analysis result is obtained.

[0124] Understandably, the data volume anomaly early warning device judges the detected information according to preset judgment rules. Specifically, if the data volume collection table's collection frequency is daily and the collection cycle is monthly, the preset judgment rule is: if R... T-1 >50% and R T-1 >=1.5R MAX And the first data size is NUM T-1 Greater than the historical data volume for the same period in March (NUM) MAX If the result is positive, it is considered an anomaly, and the second anomaly analysis result is obtained. Please refer to [the relevant documentation]. Figure 11 , Figure 11 A flowchart for implementing volatility feature detection is provided.

[0125] In practice, if the issuance of new services causes a significant increase in the amount of data collected on March 10, 2024, compared to the same period in previous years, please refer to... Figure 12 , Figure 12 An example diagram of anomaly in volatility characteristics is provided.

[0126] The above are only three feasible implementation methods of step S230 provided in this embodiment. This embodiment does not specifically limit the specific implementation method of step S230.

[0127] In a first feasible implementation, step S300 includes the following steps:

[0128] Obtain the initial early warning dataset;

[0129] It should be noted that the initial warning information set includes the verification date, warning category, warning sub-category, data block ID, reporting period, reporting interface person, and warning level information. The distinction between warning category and warning sub-category is to make the warning information visualization function more universal. If a new abnormal warning scenario occurs, a new warning scenario category can be added and accessed, and the data volume abnormal warning device will become a data volume abnormal warning device.

[0130] The warning information is input into the initial warning dataset to obtain the target warning dataset;

[0131] Understandably, the target early warning dataset is obtained by inputting early warning information into the initial early warning dataset. The data volume anomaly early warning device is equipped with a data reporting platform, which includes a data center. This data center connects the data volume collection tables stored in the data warehouse to the early warning information set, thus obtaining the target early warning dataset.

[0132] Obtain the initial feedback dataset;

[0133] It should be noted that the data volume anomaly early warning device acquires the initial feedback dataset so that the data volume anomaly early warning device can define the initial feedback dataset to obtain the target feedback information set used to collect and report the anomaly early warning confirmation status of the interface personnel.

[0134] The initial feedback dataset is defined based on the warning information to obtain the target feedback dataset;

[0135] Understandably, after obtaining the initial feedback dataset, the data volume anomaly early warning device needs to define the initial feedback dataset. The definition of the feedback dataset includes whether it has been resolved, the early warning interception category, and the reason for the early warning. After the definition is completed, the data volume anomaly early warning device can obtain the target feedback dataset.

[0136] Establish a relationship between the target early warning dataset and the target feedback dataset to obtain a preset association relationship;

[0137] It should be noted that after obtaining the target warning dataset and the target feedback dataset, the data volume anomaly warning device only needs to associate the target warning dataset with the target feedback dataset to obtain the latest feedback results and generate a visual view of the warning report. Please refer to [link / reference]. Figure 13 , Figure 13 A visual data processing flowchart for anomaly warning reports is provided; please refer to it. Figure 14 , Figure 14 It provides a visual view of the early warning reports.

[0138] The target early warning dataset is sent to the user terminal according to the preset association relationship, so that the user terminal can input feedback information into the target feedback dataset for feedback based on the target early warning dataset.

[0139] Understandably, the data volume anomaly warning device can use the notification function of the instant messaging tool connected to the data reporting platform to distribute the collected anomalies to the corresponding contact persons to form a pending notification. After receiving the notification, the contact persons can jump to the above page through the link to register and report the anomaly, and confirm the processing of the collected anomaly warning.

[0140] In its implementation, the notification function can indicate to users how many alerts have not yet been reported and require attention. Users can then click to view details and be redirected to the page shown in the image above to confirm and provide feedback on the alerts, thus forming a closed loop in the entire process. Please refer to [link / reference]. Figure 15 , Figure 15 The notification function interface diagram is provided. This notification function page diagram also includes warning-related information, such as the reporting interface person, whether the feedback person has resolved the issue, the warning interception category, the warning reason, and the last feedback time.

[0141] This embodiment detects the fluctuation and periodicity of data volume through different judgment rules, supplements and improves the method of monitoring anomalies in data collection patterns, realizes the interpretability of anomaly analysis, and improves the accuracy and efficiency of data volume anomaly early warning; it also generates a visual report of early warning results through a data reporting platform, and pushes the early warning anomaly to users for confirmation feedback, forming an automated closed-loop feedback link, solving the pain point of manual follow-up of anomalies, and further improving the efficiency of data volume anomaly early warning.

[0142] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the data volume abnormality early warning method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0143] This application also provides a data volume anomaly early warning device, please refer to... Figure 16 The data volume anomaly early warning device includes:

[0144] Module 10 is used to obtain the data volume collection table for the current day;

[0145] Analysis module 20 is used to perform anomaly analysis on the data volume in the data volume collection table based on preset collection rules and preset volatility characteristics, and obtain anomaly analysis results;

[0146] The early warning module 30 is used to generate early warning information based on the anomaly analysis results.

[0147] Optionally, the analysis module includes:

[0148] The sample module is used to acquire a historical sample data collection table; compare the historical sample data collection table with the current day's data collection table to obtain a comparison result; based on the historical sample data collection table, perform data collection pattern detection on the current day's data collection table to obtain a first anomaly analysis result; based on the historical sample data collection table, perform fluctuation feature detection on the current day's data collection table to obtain a second anomaly analysis result; and use the first anomaly analysis result and the second anomaly analysis result as the final anomaly analysis result.

[0149] Optionally, the sample module includes:

[0150] The first anomaly module is used to obtain the first data volume in the data volume collection table for the current day and the second data volume in the historical sample data volume collection table for the previous period; if the data volume collection table for the current day and the historical sample data volume collection table for the previous period belong to the data volume collection table of the same collection period, and the second data volume is greater than the first data volume, then an anomaly is determined and the first anomaly analysis result is obtained.

[0151] The volatility module is used to calculate the sample volatility corresponding to the historical sample data volume collection table for each collection cycle within the preset time period; perform correlation calculation on the sample volatility to obtain a correlation value; if the correlation value is greater than a preset correlation threshold, it is determined that the data volume volatility of the historical sample data volume collection table for each collection cycle within the preset time period has a volatility periodicity, so as to perform anomaly detection based on the volatility periodicity and obtain a second anomaly analysis result.

[0152] The second anomaly module is used to compare the first data volume with the second data volume to obtain the target data volume volatility; calculate the maximum value of the historical sample data volume collection table within a preset time period according to the preset date calculation rules to obtain the historical data volume maximum value and historical volatility maximum value of the historical sample data volume collection table within the preset time period; use the historical data volume maximum value, the historical volatility maximum value, the target data volume volatility, and the first data volume as detection information; judge the detection information, and if the target data volume volatility is greater than a preset volatility threshold, the target data volume volatility is greater than the historical volatility maximum value by a preset multiple, and the first data volume is greater than the historical maximum value, then an anomaly is determined, and a second anomaly analysis result is obtained.

[0153] Optionally, the early warning module includes:

[0154] The feedback module is used to: acquire an initial warning dataset; input the warning information into the initial warning dataset to obtain a target warning dataset; acquire an initial feedback dataset; define the initial feedback dataset according to the warning information to obtain a target feedback dataset; send the target feedback dataset to the user terminal for the user terminal to provide feedback based on the target feedback dataset; establish a relationship between the target warning dataset and the target feedback dataset to obtain a preset association relationship; and send the target warning dataset to the user terminal according to the preset association relationship for the user terminal to input feedback information into the target feedback dataset based on the warning information.

[0155] The data volume anomaly early warning device provided in this application, employing the data volume anomaly early warning method described in the above embodiments, can solve the technical problem of data volume anomaly early warning. Compared with the prior art, the beneficial effects of the data volume anomaly early warning device provided in this application are the same as those of the data volume anomaly early warning method described in the above embodiments, and other technical features in the data volume anomaly early warning device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0156] This application provides a data volume anomaly early warning device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the data volume anomaly early warning method in the above embodiment 1.

[0157] The following is for reference. Figure 17 The diagram illustrates a structural schematic suitable for implementing a data volume anomaly warning device according to embodiments of this application. The data volume anomaly warning device in embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 17 The data volume anomaly warning device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0158] like Figure 17As shown, the data volume anomaly warning device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the data volume anomaly warning device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the data volume anomaly warning device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows data volume anomaly warning devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0159] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0160] The device provided in this application, employing the data volume anomaly early warning method described in the above embodiments, can solve the technical problem of data volume anomaly early warning. Compared with the prior art, the beneficial effects of the data volume anomaly early warning device provided in this application are the same as those of the data volume anomaly early warning method described in the above embodiments, and other technical features of the data volume anomaly early warning device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0161] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0162] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0163] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the data volume anomaly warning method in the above embodiments.

[0164] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0165] The aforementioned computer-readable storage medium may be included in the data volume anomaly early warning device; or it may exist independently and not be assembled into the data volume anomaly early warning device.

[0166] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the data volume anomaly early warning device, the data volume anomaly early warning device causes the following: to acquire the data volume collection table for the day; to perform anomaly analysis on the data volume in the data volume collection table through preset collection rules and preset volatility characteristics, and to obtain anomaly analysis results; and to generate early warning information based on the anomaly analysis results.

[0167] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0168] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0169] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0170] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described data volume anomaly warning method, thereby solving the technical problem of data volume anomaly warning. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the data volume anomaly warning method provided in the above embodiments, and will not be repeated here.

[0171] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the data volume anomaly early warning method described above.

[0172] The computer program product provided in this application can solve the technical problem of abnormal data volume warning. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the abnormal data volume warning method provided in the above embodiments, and will not be repeated here.

[0173] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for early warning of abnormal data volume, characterized in that, The data volume anomaly early warning method includes: Obtain the data collection table for the current day; Anomaly analysis is performed on the data volume in the data volume collection table using preset collection rules and preset volatility characteristics to obtain anomaly analysis results; Based on the anomaly analysis results, an early warning message is generated; The step of performing anomaly analysis on the data volume in the data volume collection table based on preset collection rules and preset volatility characteristics, and obtaining the anomaly analysis results, includes: Obtain the historical sample data collection table; Based on the historical sample data collection table, the data collection table for the current day is subjected to data collection pattern detection to obtain the first anomaly analysis result. Based on the historical sample data collection table, fluctuation characteristics are detected on the data collection table for the current day to obtain the second anomaly analysis result; The first anomaly analysis result and the second anomaly analysis result are taken as the anomaly analysis result; The historical sample data collection table includes the previous historical sample data collection table. The step of detecting the collection pattern of the current day's data collection table based on the historical sample data collection table to obtain the first anomaly analysis result includes: Obtain the first data volume from the data volume collection table for the current day and the second data volume from the previous historical sample data volume collection table; If the data volume collection table for the current day and the historical sample data volume collection table for the previous period belong to the same collection period, and the second data volume is greater than the first data volume, then an anomaly is determined and a first anomaly analysis result is obtained. The step of performing fluctuation characteristic detection on the data collection table of the current day based on the historical sample data collection table to obtain the second anomaly analysis result includes: The first data volume is compared with the second data volume to obtain the target data volume volatility. The maximum value of the historical sample data collection table within a preset time period is calculated according to the preset date calculation rules, so as to obtain the maximum value of the historical data volume and the maximum value of the historical volatility of the historical sample data collection table within the preset time period. The maximum value of the historical data volume, the maximum value of the historical volatility, the target data volume volatility, and the first data volume are used as detection information; The detection information is judged. If the volatility of the target data volume is less than or equal to a preset volatility threshold, the volatility of the target data volume is greater than or equal to the maximum value of the historical volatility by a preset multiple, and the first data volume is greater than the maximum value of the historical data volume, then an anomaly is determined, and a second anomaly analysis result is obtained.

2. The data volume anomaly early warning method as described in claim 1, characterized in that, The historical sample data includes a historical sample data collection table within a preset time period. Before the step of performing fluctuation characteristic detection on the data collection table for the current day based on the historical sample data collection table to obtain the second anomaly analysis result, the following steps are included: The sample volatility corresponding to the historical sample data collection table for each collection cycle within the preset time period is calculated. The correlation of the sample volatility is calculated to obtain a correlation value. If the correlation value is greater than a preset correlation threshold, it is determined that the volatility of the historical sample data volume collection table for each collection period within the preset time period has a periodicity, so as to perform anomaly detection based on the periodicity of the volatility and obtain a second anomaly analysis result.

3. The data volume anomaly early warning method as described in claim 1, characterized in that, The step of generating early warning information based on the anomaly analysis results includes the following: Obtain the initial early warning dataset; The warning information is input into the initial warning dataset to obtain the target warning dataset; Obtain the initial feedback dataset; The initial feedback dataset is defined based on the warning information to obtain the target feedback dataset; Establish a relationship between the target early warning dataset and the target feedback dataset to obtain a preset association relationship; The target early warning dataset is sent to the user terminal according to the preset association relationship, so that the user terminal can input feedback information into the target feedback dataset for feedback based on the target early warning dataset.

4. A data volume anomaly early warning device, used to implement the data volume anomaly early warning method according to any one of claims 1-3, characterized in that, The device includes: The acquisition module is used to obtain the data collection table for the current day; The analysis module is used to perform anomaly analysis on the data volume in the data volume collection table based on preset collection rules and preset volatility characteristics, and to obtain anomaly analysis results. The early warning module is used to generate early warning information based on the anomaly analysis results.

5. A data volume anomaly early warning device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the data volume anomaly warning method as described in any one of claims 1 to 3.

6. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the method for generating early warning information based on the anomaly analysis results as described in any one of claims 1 to 3.

7. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the data volume anomaly early warning method as described in any one of claims 1 to 3.

Images