System multistage monitoring alarm method based on enterprise WeChat

Abstract

The invention belongs to the field of communication and information processing, and provides an innovative method for monitoring authorization codes of telecom number portability services. By means of a multi-source data fusion and stream processing technology, authorization code generation and user terminal logs are converged in real time. According to a dynamic rule, single-user (no response in one minute), system local (abnormal in three minutes) and global (abnormal rate is greater than or equal to 10% or interface delay exceeds 5 seconds) abnormity are rapidly judged, and three-level alarm is accurately triggered: in the first level, an interaction card is pushed through enterprise WeChat, so that one-key processing is facilitated; the second level automatically makes a call and generates a work order; and three-stage direct fusing is carried out, and a standby node is started. Meanwhile, an exponential backoff retry (at most 3 times) and a feedback learning mechanism are adopted, the threshold value is optimized, and noise is reduced. According to the method, second-level response is achieved, the authorization code issuing success rate is increased to 99.9%, the operation and maintenance efficiency is improved by 75%, the traditional monitoring blank is filled, and stable operation of telecommunication services is powerfully guaranteed.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of communication technology and information processing, specifically relating to a real-time monitoring and alarm method for communication services based on WeChat for Enterprise, which is particularly suitable for automated monitoring and multi-level alarms interacting with external systems. Through multi-source data fusion, intelligent anomaly detection, and a hierarchical response mechanism, it solves the technical problems of poor real-time performance, high false alarm rate, and difficulty in cross-system collaboration inherent in traditional monitoring methods. Background Technology

[0002] With the development of the communications industry, the number of external interaction services has increased significantly. During the execution of these services, interaction failures not only affect the user experience but also negatively impact the service quality and operational efficiency of communications companies. Summary of the Invention

[0003] Purpose of the invention It provides a real-time, accurate, automated, and multi-dimensional monitoring and alarm method, which enables rapid notification of anomalies, intelligent root cause location, and closed-loop processing through WeChat Work, ensuring the continuity and high availability of number portability services.

[0004] Technical solution The core steps of this invention include: 1. Multi-source data acquisition and anomaly detection Log collection: The following system logs are collected in real time using a lightweight agent: Authorization code generation system: authorization code generation time, status code; SMS gateway: SMS sending request time, response status, and interface latency; User terminal: Authorization code reception time, device status.

[0005] Rule 1 (Level 1 Exception): A single user authorization code is successfully generated, but the CRM does not initiate a sending request within 1 minute; Rule 2 (Level 2 Anomaly): The SMS gateway successfully sends the message, but the user terminal does not acknowledge receipt within 3 minutes; Rule 3 (Level 3 Anomalies): Anomaly rate ≥ 10% or critical interface delay ≥ 5 seconds within the same time period.

[0006] 2. Three-level alarm triggering and response mechanism

[0007] The system pushes alarm cards to local operations and maintenance personnel via WeChat Work robot. The cards include user ID, serial number, location of abnormal process (e.g., how many authorization codes were applied for, how many were successfully issued, and how many CRM authorization codes were not received), and suggestions for handling similar historical faults (e.g., contact the corresponding system for handling).

[0008] The system automatically triggers a phone call to the on-duty supervisor, simultaneously mentions the relevant team in the enterprise WeChat group, and generates a fault work order to be assigned to the designated responsible person. It also activates the backup service node.

[0009] Trigger a company-wide broadcast notification on WeChat to push a red alert message; automatically execute circuit breaker strategies (such as rate limiting and degradation) and start the root cause analysis module to generate a fault report.

[0010] 3. Intelligent closed-loop processing Automated retry: For recoverable exceptions (such as temporary interface timeouts), the authorization code re-issuance process is automatically triggered.

[0011] Processing feedback: Operations and maintenance personnel submit processing results through WeChat Work order cards. The system records the processing time and optimizes the anomaly detection threshold and alarm level.

[0012] Knowledge base construction: A classification model is trained based on historical alarm data to filter false alarms caused by users actively canceling operations.

[0013] Innovation 1. Cross-system collaborative monitoring: Real-time correlation of multi-source data is achieved through a distributed log agent and a stream processing engine (Apache Flink).

[0014] 2. Intelligent noise reduction algorithm: Based on historical data, an anomaly classification model is trained to filter false alarms (such as user-initiated cancellation of operations).

[0015] 3. Dynamic grading strategy: Dynamically adjust alarm levels based on the scope of business impact and the duration of the fault.

[0016] 4. Deep integration with WeChat for Enterprise: Alarm messages are seamlessly integrated with the work order system, address book, and approval workflow to achieve a closed-loop process of "monitoring-notification-processing-feedback". Detailed Implementation

[0017] Example 1: Level 2 Alarm (System Local Level) 1. Scenario: A failure in the CRM authorization code synchronization interface caused abnormal authorization code distribution, with an error rate exceeding 30%.

[0018] 2. System Response and Processing Results: The system triggered a Level 2 alarm. The person on duty answered the phone and activated the emergency plan, while automatically switching to the backup service.

[0019] 1. Scenario: The SMS gateway master node failed, resulting in 60 authorization codes not being sent within 10 minutes (abnormality rate 25%).

[0020] Triggered a Level 3 alarm, and a red alert was broadcast to all employees on WeChat Work; Automatically switch to the backup gateway node and limit 50% of requests to the queue cache; The root cause analysis module generated a report: "Disk I / O abnormality in the main gateway node; hardware replacement recommended."

[0021] 3. Outcome: The operations and maintenance team completed the master node repair within 30 minutes, and services were restored to normal.

[0022] Alarm response time: The average delay from the occurrence of an anomaly to the triggering of an alarm is ≤1 minute.

[0023] Business availability: The success rate of authorization code issuance has increased from 95% to 99.9%.

[0024] Operational efficiency: Manual intervention reduced by 70%, MTTR (Mean Time To Repair) reduced from 30 minutes to 5 minutes.

Claims

1. A multi-level monitoring and alarm method based on WeChat Enterprise, characterized in that, The method comprises the following steps: 1) Multi-source data collection and exception determination: (1) Real-time acquisition of interaction logs of the authorization code generation system, CRM and user terminal through a distributed log collection agent, extraction of user ID, request time and interface state field; (2) Construction of exception determination rules based on a stream processing engine, including: Single-user level exception: the authorization code generation system generates successfully, but the CRM does not initiate a sending request within 1 minute; System local level exception: the SMS gateway sends successfully, but the user terminal does not confirm receipt within 3 minutes; System global level exception: the authorization code delivery exception rate is greater than or equal to 10% or the key interface delay is greater than or equal to 5 seconds within the same period; 2) Three-level alarm triggering and response: First-level alarm: for single-user level exception, an interactive alarm card is pushed through the WeChat robot, including user ID, exception node positioning and processing suggestions, etc.; Second-level alarm: for system local level exception, an automatic telephone call is triggered to the on-duty person, and a fault work order is generated and synchronized to the WeChat group; Third-level alarm: for system global level exception, a fuse strategy is executed and a standby service node is started, and a red warning message is broadcasted through WeChat to all employees; 3) Intelligent closed-loop processing: Exponential backoff strategy is adopted for automatic retry for interface timeout type exception, and after retry failure, the exception user is marked and manual intervention process is triggered; Based on the feedback results submitted by the operation and maintenance personnel through WeChat, the exception determination threshold is optimized, and the root cause analysis knowledge base is updated.

2. The method of claim 1, wherein, The log collection agent in step (1) supports multi-protocol adaptation, including: Collecting state logs of the authorization code generation system through HTTP API; Consuming real-time message queues of the SMS gateway through Kafka; Capturing user terminal receiving state change records through a database listening module.

3. The method of claim 1, wherein, The stream processing engine in step (1) is an Apache Flink cluster, and the exception determination rules further include a dynamic threshold adjustment mechanism: Automatically correct the timeout threshold based on historical same period business load (±20% floating); If the same type of exception is triggered for 3 times continuously, the alarm level is automatically promoted.

4. The method of claim 1, wherein, The interactive alarm card of the first-level alarm in step (2) includes: Embedded buttons for directly jumping to the real-time monitoring panel of the corresponding system; A list of historical similar fault handling schemes is generated based on knowledge base similarity matching.

5. The method of claim 1, wherein, The fault work order generation of the second-level alarm in step (2) further includes: Automatically associating the affected user list and interface performance indicators (such as delay, error rate); Through the WeChat approval flow, the work order is distributed to the designated person in charge, and a processing timeout reminder (default 30 minutes) is set.

6. The method of claim 1, wherein, The fuse strategy of the third-level alarm in step (2) includes: Limiting the traffic of the SMS gateway master node by 50%, and distributing the remaining requests to the standby node; If the standby node load exceeds 80%, automatically discard low priority requests and record the compensation queue.

7. The method of claim 1, wherein, The automatic retry strategy in step (3) is as follows: The first retry interval is 1 second, and the subsequent retry interval is increased by a power of 2 (1s, 2s, 4s…), with a maximum of 3 retries; After retrying successfully, mark the user state as "repaired" and update it to the billing system synchronously.

8. The method of claim 1, wherein, Also includes an intelligent noise reduction module for: Based on the convolutional neural network (CNN) model to identify invalid logs generated by user-initiated cancellation operations; Combined with the characteristics of business periods (such as peak hours, system maintenance windows), dynamically suppress non-urgent alarms.

9. The method of claim 1, wherein, The integration method of the method and WeChat for enterprises includes: Synchronize enterprise organizational structure through OAuth 2.0 protocol to realize alarm push by region and responsibility; Support alarm message and WeChat calendar management linkage to automatically schedule fault review meetings.