Closed-loop management method and system about service multistage early warning

Abstract

The invention belongs to the technical field of service data monitoring, and particularly relates to a closed-loop management method and system about service multistage early warning. The closed-loop management method comprises the steps of data acquisition and preprocessing; cleaning and synchronizing data; monitoring a service index in real time, and triggering an early warning mechanism when the service index reaches an early warning threshold value; periodic alarm rechecking and processing: periodically tracking alarm information; and closed-loop early warning management: the closed-loop management of early warning is realized through short message interaction, automatic verification and manual marking. The closed-loop management system comprises an automatic data acquisition module, a data cleaning and synchronization module, a real-time business index monitoring and early warning module, a periodic alarm review module and a closed-loop early warning management mechanism module. According to the method and the system, the problems of false alarm or missing alarm, low response speed and lack of a closed-loop management mechanism in a conventional monitoring means are solved, comprehensive, accurate and real-time monitoring and efficient processing of the service operation state are realized, and the cost and the error rate of manual intervention are reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of business data monitoring, and in particular relates to a closed-loop management method and system for multi-level business warning. Background Art

[0002] Business monitoring refers to the real-time monitoring of business and processes based on events, messages, and services, which is crucial to the daily operations of enterprises. Although enterprises have adopted some conventional means in business monitoring, such as regular inspections, regular reports, and simple threshold alarms, these methods still have many shortcomings in practical applications:

[0003] 1) False alarms or missed alarms: Although simple threshold alarms can trigger alarms, the alarm rules are simple, prone to false alarms or missed alarms, and lack support for subsequent processing procedures. Simple threshold alarms rely on preset fixed thresholds and fail to take into account the dynamic and cyclical changes in business operations, resulting in unnecessary alarms being triggered even when the business fluctuates normally, increasing the burden on the operation and maintenance team. For subtle anomalies or potential risks in complex business scenarios, a single threshold is difficult to fully capture, and important issues are easily overlooked until the problem becomes serious. Even if the alarm is correctly triggered, if there is a lack of automated subsequent processing procedures (such as automatic diagnosis, preliminary repair attempts, etc.), manual intervention is still required, increasing response time and cost.

[0004] 2) Slow response: Conventional monitoring methods often cannot achieve real-time monitoring and timely response. When business problems occur, it often takes a long time to be discovered and handled, resulting in the expansion of problems and increased losses. Regular inspections and reports have time lags and cannot immediately reflect changes in business status, especially in high concurrency or emergencies. This lag may have serious consequences. Even if problems are discovered, manual evaluation, decision-making and response are required. Delays in any link in this process will prolong the time to solve the problem.

[0005] 3) Limited monitoring scope: Due to the limitations of manual inspections and regular reports, conventional monitoring methods can only cover some key business links and cannot fully monitor the business operation status. This may cause some potential problems to be ignored, leading to greater risks. Manual inspections often focus on known key points and easily ignore the monitoring of emerging business lines or edge systems, forming monitoring blind spots. As the scale of business expands and the complexity increases, it becomes increasingly difficult to fully monitor all business links, requiring more intelligent and automated monitoring solutions.

[0006] 4) Insufficient data processing capabilities: Conventional monitoring methods are insufficient in data processing. Due to the diversity of data collection sources, non-uniform data formats, and complex cleaning processes, data distortion or omissions are likely to occur. Traditional data analysis methods are mostly based on statistics and trend analysis, which makes it difficult to capture deep-level rules and abnormal patterns in business operations, limiting the accuracy and predictability of monitoring.

[0007] 5) Lack of closed-loop management mechanism: Conventional monitoring methods often lack a complete closed-loop management mechanism. When an alarm is triggered, there is a lack of clear processing procedures and division of responsibilities, resulting in inefficient problem handling. At the same time, there is also a lack of tracking and verification mechanisms for problem handling results, which cannot ensure that the problem is completely resolved. Summary of the invention

[0008] In order to solve the above technical problems, conventional monitoring methods have false alarms or missed alarms, slow response speed, and lack of closed-loop management mechanism, resulting in low problem handling efficiency and high labor and time costs.

[0009] The present invention provides the following technical solutions:

[0010] First aspect:

[0011] The present invention provides a closed-loop management method for multi-level early warning of business, comprising the following steps:

[0012] Step S1, data collection and preprocessing: obtaining real-time and historical business data from each business system, and performing preliminary preprocessing on the obtained business data;

[0013] Step S2, data cleaning and synchronization: receiving the original business data collected in step S1, cleaning the original business data according to preset data processing and cleaning rules, and outputting the cleaned business data;

[0014] Step S3, real-time business indicator monitoring and early warning: classify the business indicators in the monitored business data, monitor the business indicators in real time, and when the business indicators reach the early warning threshold, trigger the early warning mechanism and notify relevant personnel to respond and take measures;

[0015] Step S4, periodic alarm review and processing: regularly initiate review requests, generate review reports, and provide intelligent recommendation functions to assist reviewers in solving problems;

[0016] Step S5, closed-loop early warning management: closed-loop management of early warning is achieved through SMS interaction, automatic verification and manual marking to ensure that all alarms are effectively processed and tracked; detailed reports and visual analysis functions are provided through the early warning management report module to support managers in decision-making analysis and business optimization.

[0017] Furthermore, the data collection and preprocessing of step S1 includes:

[0018] Users set the monitoring frequency and data source type of business data according to business needs;

[0019] According to the set monitoring frequency and data source type, collect business data from the set data source;

[0020] During the data collection process, a data cache mechanism is introduced to cache intermediate business data; at the same time, exceptions in the data collection process are captured and processed; error logs are recorded;

[0021] Pre-process the collected business data, including cleaning, conversion and standardization;

[0022] The preprocessed business data is stored in the specified database or data warehouse, and indexed to support queries.

[0023] Furthermore, the data cleaning and synchronization in step S2 includes:

[0024] Build a data cleaning rule base;

[0025] Receive raw business data input and perform data cleaning operations according to preset data cleaning rules;

[0026] Use the Spark processing engine to load business data in real time and execute cleaning rules on business data in batches; monitor the data cleaning process and record processing logs;

[0027] Perform data verification before data synchronization, and use incremental synchronization or full synchronization strategies to synchronize the cleaned data from the temporary storage area to the data center or analysis platform; provide a synchronization task monitoring interface to display key indicators including synchronization progress and success rate.

[0028] Furthermore, the real-time business indicator monitoring and early warning of step S3 includes:

[0029] Use a hierarchical approach to divide business data into multiple layers;

[0030] At each level, design corresponding calculation logic and rules according to business requirements to process and analyze the data cleaned in step S2;

[0031] According to business needs and historical data, set multi-level warning thresholds for different business key indicators. The warning thresholds are fixed values ​​or dynamic thresholds calculated based on historical data.

[0032] Accurately monitor the core indicators in business data. Once the core indicator abnormality exceeds the preset trigger warning conditions, the system generates warning information, which usually includes key information such as the name of the abnormal indicator, current value, warning threshold, and abnormal occurrence time;

[0033] The generated warning information will be sent to relevant personnel to remind them to pay attention and handle it immediately; after receiving the warning notification, relevant personnel will pay attention to and handle the abnormal situation, and can take corresponding measures according to the prompts and suggestions in the warning information.

[0034] Furthermore, the periodic alarm review and processing of step S4 includes:

[0035] Initiate review requests regularly to review historical alarm records that have not been closed; and determine whether the early warning process needs to be restarted based on preset rules;

[0036] When an unresolved alarm or an alarm whose status has not been updated is retrieved, it is determined that the alarm process needs to be restarted, and the above step S3 is re-executed to calculate whether the alarm under monitoring reaches the alarm threshold; if the alarm threshold is reached, the notification system is triggered, and the notification system selects a higher level notification method to send alarm information to relevant personnel, and updates the alarm status to "under review" or a similar status;

[0037] Generate review reports regularly or on demand, summarize alarm data and provide formatted output;

[0038] Build and maintain an alarm processing knowledge base. When reviewing alarms, the intelligent recommendation engine automatically retrieves and displays relevant processing suggestions or automated scripts based on the detailed information of the alarm, helping reviewers to quickly locate problems and speed up the resolution process.

[0039] Furthermore, the closed-loop early warning management of step S5 includes:

[0040] The user communicates with the system through text messages; after receiving the warning text message sent by the system, the user responds to the warning text message;

[0041] Based on user feedback or when preset verification conditions are met, the verification process is triggered to compare business data with preset conditions to verify the problem status;

[0042] If the verification is successful, the warning status is updated to closed; if the verification fails or the user does not provide timely feedback, wait for the user to manually mark the processing status through the system interface and execute step S3;

[0043] Regularly collect warning data, including trigger time, processing time, closing time, and processing status, and generate reports to display the entire process of warning triggering, processing, and closing; provide report export and online viewing functions to support managers in decision-making analysis.

[0044] Furthermore, the business data sources to be obtained are set to be the customer service system and the transaction system, the core business indicators to be monitored are the complaint types, and the monitoring frequency is daily.

[0045] Furthermore, the hierarchical monitoring indicators to be monitored include: complaint volume threshold warning, complaint processing time warning, and complaint volume threshold warning calculation formula:

[0046]

[0047] When the ratio of the warning threshold to the average number of front-line complaints in the past 90 days is ∈[1.3,1.5), a first-level alarm is triggered; when the ratio of the warning threshold to the average number of front-line complaints in the past 90 days is ∈[1.5,2), a second-level alarm is triggered; when the ratio of the warning threshold to the average number of front-line complaints in the past 90 days is ≥2, a third-level alarm is triggered;

[0048] Complaint handling time limit warning calculation formula:

[0049]

[0050] When the ratio of the average processing time to the average processing time of the past 90 days is ∈[1.3,1.5), a level 1 alarm is triggered; when the ratio of the average processing time to the average processing time of the past 90 days is ∈[1.5,2), a level 2 alarm is triggered; when the ratio of the average processing time to the average processing time of the past 90 days is ≥2, a level 3 alarm is triggered; the past 90 days refers to 90 consecutive calendar days counted back from the current date.

[0051] Second,

[0052] The present invention also provides a closed-loop management system for multi-level business warning, which is used to execute the aforementioned closed-loop management method, including an automated data acquisition module, a data cleaning and synchronization module, a real-time business indicator monitoring and warning module, a periodic alarm review module and a closed-loop warning management mechanism module.

[0053] Automated data collection module, which is responsible for automatically collecting, preprocessing and storing business data from various data sources, including transaction data, service logs, and system performance parameters;

[0054] Data cleaning and synchronization module, used to timely transmit cleaned data to the data center or analysis platform according to preset data cleaning rules;

[0055] Real-time business indicator monitoring and early warning module, which is used to timely detect and handle abnormal situations through real-time monitoring of business indicators, generate multi-level early warning information and notify personnel to handle;

[0056] Periodic alarm review module, used to review all historical alarm records that have not been closed;

[0057] The closed-loop early warning management mechanism module is used to manage and track the entire process from the triggering to the closing of early warning information.

[0058] Furthermore, the periodic alarm review module includes a scheduled task manager, a review report generation component, a knowledge base and automatic suggestion component, a review rule engine, and an intelligent recommendation engine; the scheduled task manager is responsible for initiating requests regularly to review historical alarm records that have not been closed; the review report generation component is used to generate review reports on a scheduled or on-demand basis; when an unresolved alarm or an alarm whose status has not been updated is retrieved, the review rule engine determines whether the early warning process needs to be restarted based on preset rules; if a restart is required, the notification system is triggered, and a higher level notification method is selected to send alarm information to relevant personnel, and the alarm status is updated to "under review" or a similar status; the review report generation component generates review reports on a scheduled or on-demand basis, and summarizes the alarm data through the data analysis module.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] (1) The present invention realizes comprehensive, accurate, real-time monitoring and efficient processing of business operation status through data collection and preprocessing, data cleaning and synchronization, real-time business indicator monitoring and early warning, periodic alarm review and closed-loop early warning management, realizes comprehensive monitoring and intelligent management of business operations, and reduces the cost and error rate of manual intervention. It avoids the problems of false alarms or missed alarms, slow response speed, lack of closed-loop management mechanism, low problem handling efficiency, high labor cost and time cost caused by conventional monitoring means.

[0061] The multi-level early warning mechanism is combined with periodic review to enhance response capabilities, ensure that problems receive continuous attention and effective handling, and improve the stability of business operations. Through the closed-loop early warning management mechanism, all alarms can be managed and tracked comprehensively and meticulously until the problem is completely resolved. The strict data cleaning process ensures the quality and accuracy of the data, providing a reliable data foundation for subsequent analysis and application.

[0062] The closed-loop management method for multi-level business early warning provided by the present invention can effectively improve the monitoring capability and response efficiency of enterprise business operations, reduce operational risks, and has significant technical progress and practical value.

[0063] (2) The present invention improves the warning efficiency by setting different levels of warning thresholds and trigger conditions, setting warning thresholds, dividing warning levels, triggering conditions and countermeasures for warnings at different levels, etc.: through automated data collection and real-time indicator monitoring, rapid perception and warning of business anomalies can be achieved, reducing manual monitoring costs. It can more accurately identify potential risks in business operations and take different countermeasures according to the risk level. This mechanism helps to improve the pertinence and effectiveness of warnings.

[0064] (3) The present invention uses periodic alarm review, which is a key link to ensure that problems receive continuous attention and are handled. The multi-level early warning mechanism is combined with the periodic review to enhance the response capability, ensure that problems receive continuous attention and are effectively handled, and improve the stability of business operations. Automatically review unresolved alarms at specific time points. This mechanism ensures that problems will not be ignored due to long-term unhandled problems, thereby improving the continuity and stability of business operations. Through periodic reviews, the system can continuously track the status of problems and upgrade the warning level when necessary to attract higher levels of attention and handling.

[0065] (4) The present invention implements closed-loop early warning management through the feedback mechanism and verification process of the closed-loop early warning management. Through the closed-loop early warning management mechanism, it is ensured that all alarms can be managed and tracked comprehensively and meticulously until the problem is completely solved. A closed-loop management process from early warning triggering to problem solving is implemented. Through the preset feedback mechanism and verification process, it is ensured that all alarms can be managed and tracked comprehensively and meticulously until the problem is completely solved. This closed-loop management method helps to improve the efficiency and accuracy of problem handling and reduce operational risks.

[0066] (5) The present invention improves data quality: a strict data cleaning process ensures the quality and accuracy of the data, providing a reliable data basis for subsequent analysis and application.

[0067] (6) The present invention builds an efficient, intelligent and comprehensive closed-loop management system for multi-level business warnings, which can break through the limitations of traditional monitoring methods in terms of response speed, monitoring coverage and processing efficiency, and realize all-round and real-time monitoring of enterprise business operations. It can not only quickly capture subtle anomalies in business operations, but also predict potential risks through intelligent analysis, thus buying valuable response time for enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 The present invention is a flowchart of the closed-loop management method for multi-level business warning. DETAILED DESCRIPTION

[0069] The technical solution of the present invention will be clearly described below in conjunction with the accompanying drawings. Obviously, the described embodiments are not all embodiments of the present invention, and all other embodiments obtained by ordinary technicians in the field without making creative work are within the protection scope of the present invention.

[0070] The data targeted by this embodiment is the business data of the enterprise. The business data comes from the business system, and the business data includes but is not limited to transaction data, service logs, system performance parameters, etc. Business systems include customer service systems, transaction systems, etc.

[0071] Example 1

[0072] like Figure 1 As shown, the present invention provides a closed-loop management method for multi-level business warning, which constructs an efficient and intelligent business monitoring and management system through data collection and preprocessing, data cleaning and synchronization, real-time business indicator monitoring and warning, periodic alarm review and processing, and closed-loop warning management.

[0073] The closed-loop management method comprises:

[0074] Step S1, data collection and preprocessing: obtaining real-time and historical business data from each business system, and performing preliminary preprocessing on the obtained business data;

[0075] Specifically, step S1 includes:

[0076] Users set the monitoring frequency and data source type of business data (such as databases, log files, API interfaces, and other sources) according to business needs;

[0077] Collect business data from the set data source according to the set monitoring frequency and data source type; such business data includes but is not limited to transaction data, service logs, system performance parameters, etc.;

[0078] During the data collection process, a data cache mechanism (such as using an in-memory database such as Redis) is introduced to cache intermediate business data. At the same time, exceptions in the data collection process, such as network problems and insufficient permissions, are captured and processed. Error logs are recorded to facilitate subsequent troubleshooting and repair of problems.

[0079] Preprocess the collected data, including cleaning, conversion and standardization;

[0080] The preprocessed business data is stored in the specified database or data warehouse, and indexed to support fast query.

[0081] To give a specific example:

[0082] For the first-line complaint classification warning scenario:

[0083] First, clarify the types of complaints that need to be monitored (i.e. specific business data), such as product quality complaints, service attitude complaints, etc.

[0084] Secondly, the business systems that need to be monitored are set as customer service system and transaction system, and the monitoring frequency is daily.

[0085] Obtain relevant business data from various business systems, for example: obtain transaction data, through which you can analyze whether users have complaints after purchase; obtain service logs, through which you can monitor the processing status and response time of customer service personnel; obtain system performance parameters, through which you can evaluate whether the system has caused an increase in user complaints due to failures.

[0086] Step S2, data cleaning and synchronization: receiving the original business data collected in step S1, cleaning the original business data according to preset data processing and cleaning rules, and outputting the cleaned business data.

[0087] Business data cleaning includes deduplication, denoising, format conversion, missing value processing, and sensitive word desensitization. Deduplication: For example, detecting and identifying duplicate complaint records in the original business data, usually based on the unique identifier of the complaint (such as complaint number, customer ID, etc.), and retaining the latest record.

[0088] Denoising: Identify and remove invalid data, such as identifying and deleting garbled characters, irrelevant complaints and other invalid data.

[0089] Format conversion: For example, the date format can be uniformly converted to the YYYY-MM-DD format to ensure the consistency and readability of date data. At the same time, other data types (such as time, numbers, etc.) can also be converted.

[0090] Missing value processing: such as filling in missing complaint types or customer information, or deleting records with missing data.

[0091] Sensitive word desensitization: Identify sensitive words and build a sensitive word library, including personal information (such as name, phone number), confidential information, etc. Replace sensitive words with "***" or "User 1" to ensure that the data no longer contains identifiable personal information.

[0092] Step S2 includes:

[0093] Build a data cleaning rule base and be able to modify the data cleaning rule base;

[0094] Receive raw business data input and perform cleaning operations such as deduplication, denoising, format conversion, and missing value processing according to preset data cleaning rules;

[0095] Use the Spark processing engine to load business data in real time and execute cleaning rules on business data in batches; monitor the data cleaning process and record processing logs;

[0096] Perform data verification before data synchronization, and use incremental synchronization or full synchronization strategies to synchronize the cleaned data from the temporary storage area to the data center or analysis platform; provide a synchronization task monitoring interface to display key indicators such as synchronization progress and success rate.

[0097] Through step S2, the original business data can be fully cleaned and efficiently synchronized, providing high-quality data support for subsequent data analysis and decision-making. At the same time, by continuously optimizing and improving the data cleaning rule base and synchronization strategy, the efficiency and accuracy of data processing can be further improved.

[0098] Step S3, real-time business indicator monitoring and early warning: classify the monitoring indicators, monitor the business indicators in real time, and when the business indicators reach the early warning threshold, trigger the early warning mechanism and notify relevant personnel to respond and take measures.

[0099] Step S3 comprises,

[0100] Using a hierarchical approach, business data is divided into layers such as ODS (operational data storage) layer, DWD (data warehouse detail layer), and DWS (data warehouse summary layer);

[0101] At each level, design corresponding calculation logic and rules according to business requirements to process and analyze the data cleaned in step S2;

[0102] According to business needs and historical data, set multi-level warning thresholds for different business key indicators. The thresholds can be fixed values ​​or dynamic thresholds calculated based on historical data.

[0103] Implement precise monitoring of core indicators. Once an indicator exceeds the preset trigger warning condition, the system automatically generates warning information, which usually includes key information such as the name of the abnormal indicator, current value, warning threshold, and abnormal occurrence time;

[0104] The generated warning information will be sent to relevant personnel. The notification content will clearly inform the severity and urgency of the abnormal situation, and remind relevant personnel to pay attention and handle it immediately. After receiving the warning notification, relevant personnel will immediately pay attention to and handle the abnormal situation, and can take corresponding measures according to the prompts and suggestions in the warning information.

[0105] The system will record the processing and results of the warning information, which can be tracked and analyzed later to optimize the warning strategy and response process.

[0106] Through steps such as data stratification and classification, warning threshold setting, precise monitoring and warning, warning information generation and emergency notification sending, as well as response and tracking, the accuracy and timeliness of warnings are ensured, providing strong support for the company's business operations.

[0107] In business, the commonly used hierarchical monitoring indicators are: complaint volume threshold warning, complaint processing time warning, and complaint volume threshold warning calculation formula:

[0108] Complaint volume threshold warning calculation formula:

[0109]

[0110] When the warning threshold is between 1.3 (inclusive) and 1.5 (exclusive) times the average of the number of front-line complaints in the past 90 days, a level one alarm is triggered; when the warning threshold is between 1.5 (inclusive) and 2 (exclusive) times the average of the number of front-line complaints in the past 90 days, a level two alarm is triggered; when the warning threshold is more than 2 (inclusive) times the average of the number of front-line complaints in the past 90 days, a level three alarm is triggered.

[0111] Complaint handling time limit warning calculation formula:

[0112]

[0113] When the average processing time is 1.3 (inclusive) to 1.5 (exclusive) times the average processing time of the past 90 days, a level 1 alarm is triggered; when the average processing time is 1.5 (inclusive) to 2 (exclusive) times the average processing time of the past 90 days, a level 2 alarm is triggered; when the average processing time is more than 2 (inclusive) times the average processing time of the past 90 days, a level 3 alarm is triggered. The past 90 days refers to 90 consecutive calendar days counted back from the current date.

[0114] Step S4, periodic alarm review and processing: regularly initiate review requests, generate review reports, and provide intelligent recommendation functions to help reviewers quickly solve problems.

[0115] Step S4 includes:

[0116] Initiate review requests regularly to review historical alarm records that have not been closed: By setting the alarm review cycle (such as 16 hours, 24 hours, 48 ​​hours), the review task is automatically triggered to review historical alarm records that have not been closed; and determine whether the early warning process needs to be restarted based on preset rules;

[0117] When an unresolved alarm or an alarm whose status is not updated is retrieved, it is determined that the alarm process needs to be restarted, and the above step S3 is re-executed to calculate whether the alarm under monitoring reaches the alarm threshold; if the alarm threshold is reached, the notification system is triggered, and the notification system selects a higher level notification method (such as a phone call) to send an alarm message to relevant personnel, and updates the alarm status to "under review" or a similar status;

[0118] Generate review reports regularly or on demand, summarize alarm data and provide formatted output: Use the data analysis module to process alarm data, generate formatted reports through the report template engine, and support exporting reports to PDF, Excel and other formats for management personnel to review and evaluate;

[0119] Build and maintain an alarm processing knowledge base, including common problems, solutions, and automated scripts. When reviewing an alarm, the intelligent recommendation engine automatically retrieves and displays relevant processing suggestions or automated scripts based on the detailed information of the alarm, helping reviewers to quickly locate the problem and speed up the resolution process. Reviewers can quickly locate the problem and speed up the resolution process based on the recommended processing suggestions or automated scripts.

[0120] Step S5, closed-loop early warning management: closed-loop management of early warning is achieved through functions such as SMS interaction, automatic verification and manual marking to ensure that all alarms are effectively processed and tracked; detailed reports and visual analysis functions are provided through the early warning management report module to support managers in decision-making analysis and business optimization.

[0121] Step S5 includes:

[0122] Users communicate with the system through text messages. After receiving the warning text message sent by the system, users can provide feedback through simple replies (such as "Y" to indicate resolved) to analyze the content of user feedback;

[0123] Based on user feedback or when preset verification conditions are met, the verification process is triggered to compare business data with preset conditions to verify the problem status;

[0124] If the verification is successful, the warning status is updated to closed; if the verification fails or the user does not provide timely feedback, wait for the user to manually mark the processing status through the system interface and execute step S3; and update the warning status;

[0125] Regularly collect warning data, including trigger time, processing time, closing time, processing status, etc., generate reports to display the entire process of warning triggering, processing, and closing; provide report export and online viewing functions to support managers in decision-making analysis.

[0126] It realizes the automatic sending, receiving, verification and processing of early warning information, as well as the collection, report generation and analysis of early warning data, ensuring that all alarms are effectively tracked and managed, and improving the efficiency and accuracy of early warning management.

[0127] Example 2

[0128] This embodiment provides a closed-loop management system for multi-level business warnings, which integrates multiple modules such as an automated data collection module, a data cleaning and synchronization module, a real-time business indicator monitoring and warning module, a periodic alarm review module, and a closed-loop warning management mechanism module.

[0129] The automated data collection module is responsible for automatically collecting, preprocessing and storing business data from various data sources, including transaction data, service logs, system performance parameters, etc. The automated data collection module includes a scheduled task manager, a data source configuration module, a data collection engine, a data cache system, a data processing and conversion module, and a data storage and indexing module. This module achieves efficient, accurate and reliable data collection through a series of components working together.

[0130] The scheduled task manager is responsible for scheduling and triggering data collection tasks. It automatically triggers data collection tasks according to the time interval and trigger conditions configured by the user. By using the Quartz scheduled task framework, a flexible scheduled task configuration interface is provided. Users can configure the time interval of data collection tasks through the scheduled task configuration interface to collect data from the data source every day or every cycle.

[0131] The data source configuration module is used to provide a user interface (UI) to configure parameters such as data source and monitoring frequency. It supports the configuration of multiple data source types (such as databases, log files, API interfaces, etc.). The monitoring frequency of the data source configuration module focuses on the collected data and calculates and verifies the data according to the time window.

[0132] The data collection engine is used to execute the operation of obtaining business data from the configured data source according to the configuration and rules preset by the data source configuration module.

[0133] These data sources can be databases, log files, API interfaces, sensors, exception handling, etc. Database: Connect to the database through JDBC and execute SQL query statements to obtain data. Log files: Use log parsing libraries (such as Logback, Log4j2's AppenderAPI or custom parsers) to read and parse log files. API interface: Use HTTP clients (such as RestTemplate, HttpClient) to call RESTfulAPI to obtain data in JSON or XML format. Exception handling: Capture and handle exceptions in the data collection process, such as network problems, insufficient permissions, etc., and record error logs.

[0134] The data cache system is used to cache intermediate data and reduce the pressure of direct access to the source system.

[0135] Use memory databases such as Redis as cache storage to store frequently accessed hot data and reduce the number of database accesses. Set reasonable cache partitioning and elimination strategies based on the access frequency and importance of the data. At the same time, you can preheat the cache during the system off-peak period, preload and cache key data to improve the response speed during peak periods.

[0136] Data processing and conversion module, used to clean, convert and standardize the collected data;

[0137] The data storage and indexing module is used to store the data processed by the data processing and conversion module into a specified database or data warehouse, and to create indexes to support fast query.

[0138] By implementing the above-mentioned automated data collection module, enterprises can collect, process and store business data more efficiently, providing strong support for business analysis and decision-making.

[0139] The data cleaning and synchronization module is used to improve data quality through preset data cleaning rules, and transmit the cleaned data to the data center or analysis platform in a timely manner through efficient data synchronization tools. This module includes components such as data cleaning rule engine, data cleaning rule base, rule base management system, batch processing framework and data synchronization tool.

[0140] The data cleaning rule engine is used to execute the preset data cleaning rules. It receives the original data, performs operations such as deduplication, denoising, format conversion, and missing value processing according to the preset data cleaning rule library, and outputs the cleaned data.

[0141] The data cleaning rule library includes but is not limited to deduplication, denoising, format conversion, missing value processing, etc., to improve data quality. Deduplication rules: define deduplication strategies based on unique keys (such as ID, timestamp + specific field combinations), and use technologies such as hash tables or database indexes to achieve efficient deduplication. Denoising rules: use regular expression matching, statistical anomaly detection and other methods to identify and remove noise in the data, such as outliers, invalid characters, etc. Format conversion rules: define unified standards for data formats, such as date and time formats, numeric formats, etc., and provide conversion functions or templates for automated processing. Missing value processing rules: set a filling strategy for missing values, such as using the mean, median, or mode, or performing special processing based on business logic.

[0142] The rule base management system is used to manage and maintain the data cleaning rule base. The rule base management system provides a user interface that allows users to configure, add, modify and delete cleaning rules.

[0143] Batch processing framework, used to process large-scale batch data using Spark. Spark has excellent parallel processing capabilities and can significantly improve the efficiency of data processing. Compared with traditional big data processing frameworks, Spark has lower latency and is more suitable for real-time data processing scenarios. Spark provides rich library support, such as SparkSQL (for structured data processing) and MLlib (for machine learning), which facilitates data analysis and mining.

[0144] Load business data into the Spark processing engine, execute the rules in the data cleaning rule base, and clean the collected data in real time or quasi-real time to ensure data quality. In addition, during the monitoring data cleaning process, record the processing log for troubleshooting and performance optimization.

[0145] Data synchronization tool, used to synchronize cleaned data from temporary storage area to data center or analysis platform. Implement data verification mechanism to ensure data integrity and accuracy during synchronization. Optimize the synchronization process, such as using parallel processing, compressed transmission and other technologies to improve synchronization efficiency. Provide synchronization task monitoring interface, display key indicators such as synchronization progress and success rate, and support manual triggering or automatic scheduling of synchronization tasks.

[0146] The data synchronization tool has efficient and reliable data transmission capabilities, and supports two strategies: incremental synchronization and full synchronization. For real-time or quasi-real-time cleansed data, the incremental synchronization strategy is adopted to synchronize only newly added or changed data. In the case of initial synchronization or major changes in data structure, the full synchronization strategy is adopted to ensure data integrity and consistency.

[0147] The real-time business indicator monitoring and early warning module is used to monitor business indicators in real time, promptly detect and handle abnormal situations, and ensure the stability and efficiency of business operations. This module includes an early warning strategy engine, notification and response system.

[0148] The early warning strategy engine is used to set multi-level early warning thresholds and monitor business indicators in real time. Once a business indicator triggers an early warning condition, the early warning strategy engine will automatically generate early warning information.

[0149] The early warning strategy engine divides business data into: ODS (operational data storage) layer, DWD (data warehouse detail layer) layer, and DWS (data warehouse summary layer) layer.

[0150] The ODS (operational data storage) layer stores the original data collected from the business system. Generally, it only checks whether the data is extracted normally and whether there is any missing data. Generally, there will only be a data missing alarm.

[0151] The DWD layer cleans, converts, and normalizes the data in the ODS layer to generate basic data that can be used for subsequent analysis. The DWD layer stores data that has been cleaned, converted, and formatted. Most of the alarms described below are also for this layer, checking whether a certain value exceeds the threshold.

[0152] The DWS layer further summarizes and calculates the data in the DWD layer to generate various indicators and report data required for the business. The DWS layer stores statistical report data, such as daily reports, weekly reports, monthly reports, etc., mainly for year-on-year and month-on-month calculations, or alarms when the score is below the threshold.

[0153] At each level, corresponding calculation logic and rules are designed according to business needs to process and analyze the cleaned data. Through the layered and graded approach, the accuracy and efficiency of the calculation results can be ensured, while facilitating subsequent data management and query.

[0154] According to the business needs and historical data of the enterprise, set multi-level warning thresholds for different key business indicators. These thresholds can be fixed values ​​or dynamic thresholds calculated based on historical data (such as the average value of the past 30 days minus two standard deviations). The setting of warning thresholds needs to fully consider the actual situation and tolerance of the business to ensure the accuracy and effectiveness of the warning.

[0155] Accurately monitor the core indicators such as timeliness in business indicators. These core indicators are crucial to the business operations of the enterprise, so they need special attention. By monitoring the changes of these core indicators in real time and comparing them with the set warning thresholds, once the warning conditions are triggered, warning information is immediately generated and the notification and response system is triggered. These warning information usually includes key information such as the name of the abnormal business indicator, current value, warning threshold, and abnormal occurrence time. The warning information will also be formatted according to the preset template so that relevant personnel can quickly understand the abnormal situation.

[0156] The notification and response system is used to send warning information to relevant personnel in a timely manner and track the response status through various notification methods such as SMS, email, and APP push. These notification methods can be selected according to personnel preferences and actual conditions. The notification content will clearly inform the severity and urgency of the abnormal situation, reminding relevant personnel to pay attention and handle it immediately.

[0157] After receiving the early warning notification, relevant personnel will immediately pay attention to and handle the abnormal situation. They can take corresponding measures based on the prompts and suggestions in the early warning information. The notification and response system will also record the processing and results of the early warning information for subsequent tracking and analysis. This helps enterprises continuously optimize early warning strategies and response processes, and improve the stability and efficiency of business operations.

[0158] The periodic alarm review module is used to ensure that all open historical alarm records are reviewed in a timely and effective manner, including a scheduled task manager, a review report generation component, a knowledge base and automatic suggestion component, a review rule engine, and an intelligent recommendation engine.

[0159] The scheduled task manager is responsible for initiating requests regularly and reviewing historical alarm records that have not been closed.

[0160] Automatically trigger the review task according to the preset time interval. Set the alarm review cycle (such as 16 hours, 24 hours, 48 ​​hours), and the scheduled task manager will automatically initiate a request to review the historical alarm records that have not been closed. Through regular review, you can promptly discover and handle those alarms that have been missed or have not been resolved for a long time, improving the stability and reliability of the system.

[0161] The review report generation component is used to generate review reports on a scheduled or on-demand basis. When an unresolved alarm or an alarm whose status has not been updated is retrieved, the review rule engine determines whether the early warning process needs to be restarted based on the preset rules. If a restart is required, the notification system is triggered, and a higher-level notification method (such as a phone call) is selected to send the alarm information to relevant personnel, and the alarm status is updated to "under review" or a similar status. Review reports are generated on a scheduled or on-demand basis, and the alarm data is summarized through the data analysis module. Use the report template engine to generate formatted reports, which can be exported to PDF, Excel and other formats for easy review and evaluation by managers.

[0162] The knowledge base and automatic suggestion components are used to build and maintain a comprehensive alarm processing knowledge base, including common problems, solutions, automated scripts, etc., and provide intelligent recommendation functions to help reviewers quickly solve problems. When reviewing alarms, the intelligent recommendation engine automatically retrieves and displays relevant processing suggestions or automated scripts based on the detailed information of the alarm, helping reviewers quickly locate problems and speed up the resolution process.

[0163] The periodic alarm review module can ensure that all open alarm records are reviewed and processed in a timely and effective manner, improving the stability and reliability of the system. At the same time, the module can also provide comprehensive alarm processing information and suggestions for operation and maintenance personnel to help them quickly locate and solve problems.

[0164] The closed-loop early warning management mechanism module is used to ensure that the entire process of early warning information from triggering to closing is effectively managed and tracked. It includes SMS interaction module, automatic verification module, status manager, and early warning management report module.

[0165] The SMS interaction module is responsible for SMS communication with users, including sending warning information to users and receiving feedback or responses from users via SMS. The SMS interaction module has an SMS receiver and an SMS parser. After the user receives the SMS sent by the notification and response system, the user is allowed to give feedback through a simple reply (such as "Y" for resolved). The SMS receiver receives the user's reply and passes it to the SMS parser, which parses the reply content and passes the parsing result to the automatic verification module.

[0166] The automatic verification module is used to automatically trigger the verification process based on user feedback or preset conditions, and compare the business data to verify the problem status. When the automatic verification module receives user feedback or meets the preset verification conditions, it triggers the verification process. Through the data interface, the automatic verification module obtains the actual business data and compares it with the preset conditions. If the verification passes (that is, the user has resolved the warning or the business data meets the preset conditions), the status updater will update the warning status to closed; if the verification fails (that is, the business data does not meet the preset conditions) or the user does not provide timely feedback, it waits for the user to manually mark the processing status through the business system interface to ensure that all alarms are effectively tracked and managed.

[0167] The state manager is used to manage the life cycle of the warning state, including the triggering, processing, and closing of the warning.

[0168] The early warning management report module is used to regularly collect early warning data, including trigger time, processing time, closing time, processing status, etc. Use the report tool to generate reports based on the collected data to show the entire process of early warning triggering, processing, and closing. Provide report export and online viewing functions to support managers in decision-making analysis.

[0169] The closed-loop early warning management mechanism module realizes the automation, standardization and visualization of early warning management by integrating functions such as SMS interaction, automatic verification, status management and report generation. The module can significantly improve the efficiency and accuracy of early warning processing, reduce the cost of manual intervention, and provide users with more reliable and efficient early warning management services. At the same time, through report display and decision analysis functions, managers can better understand the overall situation of early warning management and make more informed decisions.

[0170] The above technical features constitute the best embodiment of the present invention, which has strong adaptability and best implementation effect. Non-essential technical features can be added or reduced according to actual needs to meet the needs of different situations.

[0171] Finally, it should be noted that the above content is only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Simple modifications or equivalent substitutions of the technical solution of the present invention by ordinary technicians in this field do not deviate from the essence and scope of the technical solution of the present invention.

Claims

1. A closed-loop management method for multi-level early warning of business, characterized in that: The steps include: Step S1, data collection and preprocessing: obtaining real-time and historical business data from each business system, and performing preliminary preprocessing on the obtained business data; Step S2, data cleaning and synchronization: receiving the original business data collected in step S1, cleaning the original business data according to preset data processing and cleaning rules, and outputting the cleaned business data; Step S3, real-time business indicator monitoring and early warning: classify the business indicators in the monitored business data, monitor the business indicators in real time, and when the business indicators reach the early warning threshold, trigger the early warning mechanism and notify relevant personnel to respond and take measures; Step S4, periodic alarm review and processing: regularly initiate review requests, generate review reports, and provide intelligent recommendation functions to help reviewers solve problems; Step S5, closed-loop warning management: closed-loop management of warnings is achieved through SMS interaction, automatic verification and manual marking to ensure that all alarms are effectively processed and tracked; The early warning management report module provides detailed reports and visual analysis functions to support managers in decision-making analysis and business optimization.

2. The closed-loop management method for multi-level business warning according to claim 1 is characterized in that: The data collection and preprocessing of step S1 include: Users set the monitoring frequency and data source type of business data according to business needs; According to the set monitoring frequency and data source type, collect business data from the set data source; During the data collection process, a data cache mechanism is introduced to cache intermediate business data; at the same time, exceptions in the data collection process are captured and processed; error logs are recorded; Pre-process the collected business data, including cleaning, conversion and standardization; The preprocessed business data is stored in the specified database or data warehouse, and indexed to support queries.

3. The closed-loop management method for multi-level business warning according to claim 2 is characterized in that: The data cleaning and synchronization of step S2 includes: Build a data cleaning rule base; Receive raw business data input and perform data cleaning operations according to preset data cleaning rules; Use the Spark processing engine to load business data in real time and execute cleaning rules on business data in batches; monitor the data cleaning process and record processing logs; Perform data verification before data synchronization, and use incremental synchronization or full synchronization strategies to synchronize the cleaned data from the temporary storage area to the data center or analysis platform; provide a synchronization task monitoring interface to display key indicators including synchronization progress and success rate.

4. The closed-loop management method for multi-level business warning according to claim 3 is characterized in that: The real-time business indicator monitoring and early warning of step S3 includes: Use a hierarchical approach to divide business data into multiple layers; At each level, design corresponding calculation logic and rules according to business requirements to process and analyze the data cleaned in step S2; According to business needs and historical data, set multi-level warning thresholds for different business key indicators. The warning thresholds are fixed values ​​or dynamic thresholds calculated based on historical data. Accurately monitor the core indicators in business data. Once the core indicator abnormality exceeds the preset trigger warning conditions, the system generates warning information, which usually includes key information such as the name of the abnormal indicator, current value, warning threshold, and abnormal occurrence time; Send the generated warning information to relevant personnel to remind them to pay attention and handle it; After receiving the early warning notification, relevant personnel will pay attention to and handle abnormal situations, and can take corresponding measures based on the prompts and suggestions in the early warning information.

5. The closed-loop management method for multi-level business warning according to claim 4 is characterized in that: The periodic alarm review and processing of step S4 includes: Initiate review requests regularly to review historical alarm records that have not been closed; and determine whether the early warning process needs to be restarted based on preset rules; When an unresolved alarm or an alarm whose status has not been updated is retrieved, it is determined that the alarm process needs to be restarted, and the above step S3 is re-executed to calculate whether the alarm under monitoring reaches the alarm threshold; if the alarm threshold is reached, the notification system is triggered, and the notification system selects a higher level notification method to send alarm information to relevant personnel, and updates the alarm status to "under review" or a similar status; Generate review reports regularly or on demand, summarize alarm data and provide formatted output; Build and maintain an alarm processing knowledge base. The intelligent recommendation engine automatically retrieves and displays relevant processing suggestions or automated scripts based on the detailed information of the alarm, helping reviewers to quickly locate problems and speed up the resolution process.

6. The closed-loop management method for multi-level business warning according to claim 5 is characterized in that: The closed-loop early warning management of step S5 includes: The user communicates with the system through text messages; after receiving the warning text message sent by the system, the user responds to the warning text message; Based on user feedback or when preset verification conditions are met, the verification process is triggered to compare business data with preset conditions to verify the problem status; If the verification is successful, the warning status is updated to closed; if the verification fails or the user does not provide timely feedback, wait for the user to manually mark the processing status through the system interface and execute step S3; Regularly collect warning data, including trigger time, processing time, closing time, and processing status, and generate reports to display the entire process of warning triggering, processing, and closing; provide report export and online viewing functions to support managers in decision-making analysis.

7. The closed-loop management method for multi-level business warning according to claim 6 is characterized in that: The business data sources to be obtained are set as the customer service system and the transaction system, the core business indicator to be monitored is the complaint type, and the monitoring frequency is daily.

8. The closed-loop management method for multi-level business warning according to claim 7 is characterized in that: The hierarchical monitoring indicators to be monitored include: complaint volume threshold warning, complaint processing time warning, and complaint volume threshold warning calculation formula: When the ratio of the warning threshold to the average number of front-line complaints in the past 90 days is ∈[1.3,1.5), a first-level alarm is triggered; when the ratio of the warning threshold to the average number of front-line complaints in the past 90 days is ∈[1.5,2), a second-level alarm is triggered; when the ratio of the warning threshold to the average number of front-line complaints in the past 90 days is ≥2, a third-level alarm is triggered; Complaint handling time limit warning calculation formula: When the ratio of the average processing time to the average processing time of the past 90 days is ∈[1.3,1.5), a level 1 alarm is triggered; when the ratio of the average processing time to the average processing time of the past 90 days is ∈[1.5,2), a level 2 alarm is triggered; when the ratio of the average processing time to the average processing time of the past 90 days is ≥2, a level 3 alarm is triggered; The past 90 days refers to 90 consecutive calendar days counted back from the current date.

9. A closed-loop management system for multi-level business warning, characterized in that: Used to execute the closed-loop management method described in any one of claims 1 to 8, comprising an automated data acquisition module, a data cleaning and synchronization module, a real-time business indicator monitoring and early warning module, a periodic alarm review module and a closed-loop early warning management mechanism module, Automated data collection module, which is responsible for automatically collecting, preprocessing and storing business data from various data sources, including transaction data, service logs, and system performance parameters; Data cleaning and synchronization module, used to timely transmit cleaned data to the data center or analysis platform according to preset data cleaning rules; Real-time business indicator monitoring and early warning module, which is used to timely detect and handle abnormal situations through real-time monitoring of business indicators, generate multi-level early warning information and notify personnel to handle; Periodic alarm review module, used to review all historical alarm records that have not been closed; The closed-loop early warning management mechanism module is used to manage and track the entire process from the triggering to the closing of early warning information.

10. The closed-loop management system for multi-level business warning according to claim 9, characterized in that: The periodic alarm review module includes a scheduled task manager, a review report generation component, a knowledge base and automatic suggestion component, a review rule engine, and an intelligent recommendation engine. The scheduled task manager is responsible for initiating requests regularly to review historical alarm records that have not been closed. The review report generation component is used to generate review reports on a scheduled or on-demand basis. When an unresolved alarm or an alarm whose status has not been updated is retrieved, the review rule engine determines whether the early warning process needs to be restarted based on preset rules. If restart is required, the notification system is triggered, and a higher level notification method is selected to send alarm information to relevant personnel, and the alarm status is updated to "under review" or similar status; the review report generation component generates review reports regularly or on demand, and summarizes the alarm data through the data analysis module.

Images