Full-view monitoring system and method for application
Through the passive and active detection module combined with the data collection and analysis module, a multi-dimensional business application status evaluation model is built, which solves the problems of single perspective and delay in the traditional monitoring system in complex environments, realizes real-time and multi-angle monitoring and fault prediction, and improves the usability and stability of the system.
Patent Information
- Application Number
- CN202510826208.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-19
- Publication Date
- 2025-08-15
AI Technical Summary
Under the multi-service concurrency, high concurrency access and complex dependency chains, traditional monitoring systems have problems such as single perspective, state delay, inability to detect false deaths and lack of active detection, which makes it difficult to achieve multi-dimensional state perception, real-time response and high availability.
Passive detection module and active detection module are used to combine data collection and analysis modules and alarm modules to build a multi-dimensional business application status evaluation model by integrating process-level, service-level, network-level, and logic-level monitoring data, and perform real-time alarm and fault self-healing operations.
It realizes multi-angle, real-time monitoring of business applications, can actively detect and predict abnormalities, improves the availability and stability of the system, and ensures the high-dimensional availability of key business systems.
Smart Images
Figure CN120499052A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of application monitoring technology, and in particular to a full-view monitoring system and method for an application. Background Art
[0002] With the continuous development of information technology, the internet has become an indispensable tool in our daily lives and work. With the increasing number of internet services, microservices architecture is currently a widely used and widely used software service architecture. In today's distributed computing, cloud platforms, and edge computing environments, the operating status of business applications has a significant impact on system stability and service availability.
[0003] To ensure the stable operation of microservices, we typically have monitoring and maintenance systems to observe the application's operating status. Traditional system monitoring methods mostly rely on single-point log analysis, fixed threshold alarms, and intermittent detection mechanisms, making it difficult to fully and realistically understand the application's operating status.
[0004] Especially in situations with multiple services running concurrently, high concurrent access, and complex dependency chains, traditional monitoring methods often have the following flaws:
[0005] Single perspective: Only monitors system indicators such as CPU and memory, ignoring the interaction status between services;
[0006] Status delay: Monitoring has periodic delays and cannot achieve second-level response;
[0007] Unable to detect "fake death": The process exists but the internal logic is stuck, which traditional heartbeat mechanisms cannot identify;
[0008] Lack of active detection mechanism: The system can only record passively and lacks automatic identification and response mechanism.
[0009] Therefore, there is an urgent need for a systematic approach that can achieve multi-dimensional state perception, active activity detection, real-time response and multi-angle monitoring to ensure the high availability and stability of business systems. Summary of the Invention
[0010] In response to the above problems, the present invention provides a full-perspective monitoring system and method for applications. By integrating multi-level monitoring data such as process level, service level, network level, and logical level, a comprehensive assessment of the application running status is achieved, real-time alarms are issued for applications with abnormal status, and active detection of critical business systems is performed to achieve higher-dimensional availability of critical systems.
[0011] To achieve the above objectives, the present invention provides an application-based full-view monitoring method, comprising:
[0012] Passive detection module, active detection module, data collection and analysis module and alarm module;
[0013] The passive detection module is used to:
[0014] Integrated in business applications, receiving and, according to the settings of the data collection and analysis module, regularly sending monitoring data to the business collection and analysis module, including process-level, service-level, network-level and logic-level monitoring data;
[0015] The active detection module is used to:
[0016] Receive and, based on the settings of the data collection and analysis module, simulate real business messages to send requests to business applications, and regularly send the request information and response data as monitoring data to the data collection and analysis module;
[0017] The data collection and analysis module is used to:
[0018] Receive the monitoring data sent by the passive detection module and the active detection module, send the business application status assessment model, obtain the assessment result, and if it meets the expectations, determine that the business application operation status is normal; otherwise, determine that the operation status is abnormal, and push the abnormal business application and the corresponding monitoring data to the alarm module;
[0019] The alarm module is used to:
[0020] Send abnormal business applications and corresponding monitoring data information to operation and maintenance personnel;
[0021] Perform business application fault self-recovery operations according to the configured business fault handling rules.
[0022] As a further improvement of the present invention, the setting of the data collection and analysis module received by the passive detection module includes:
[0023] Collect information from various business applications, including logs, metrics, and service link information;
[0024] Regular detection time and detection range for each business application.
[0025] As a further improvement of the present invention, the settings of the data collection and analysis module received by the active detection module include:
[0026] The service interface, detection interval, and detection time range of each business application's active detection.
[0027] As a further improvement of the present invention, the active detection module sends the request information, response result, time consumption information and status code as monitoring data to the data collection and analysis module regularly.
[0028] As a further improvement of the present invention, the data collection and analysis module incorporates the monitoring data sent by the passive detection module and the active detection module into the business application status evaluation model respectively, and evaluates the running status of the business application respectively.
[0029] As a further improvement of the present invention, a multi-dimensional and three-dimensional business application operation status evaluation model is constructed based on historical data of business application operation.
[0030] As a further improvement of the present invention, a large application operation model is constructed based on the business application operation history data, and the large application operation model is trained to obtain a business application operation status evaluation model.
[0031] As a further improvement of the present invention, the business application operation history data includes: hardware attributes, software attributes, log attributes, indicator attributes, service link attributes and business attributes;
[0032] The hardware attributes include: CPU frequency, number, CPU instruction set, memory size, and memory frequency.
[0033] The software attributes include: operating system, JDK version, and compilation parameters.
[0034] The log attributes include: log time, log level, and log information.
[0035] The indicator attributes include: CPU usage, memory usage, request time, request response, status code, etc.
[0036] The service link attributes include: traceId, spanId, and call context tag.
[0037] The service attributes include: service name, version, runtime, service interface, service request field, and service response field.
[0038] As a further improvement of the present invention, training the application running large model includes:
[0039] Configure the business application to enable transaction recording, and completely record the business application's historical running data during the transaction period as training data.
[0040] The present invention further provides an application full-view monitoring method, using the application full-view monitoring system according to any one of claims 1 to 9, characterized in that it includes:
[0041] The passive detection module receives and regularly sends monitoring data to the data collection and analysis module according to the settings of the data collection and analysis module, including multi-level monitoring data such as process level, service level, network level, and logical level.
[0042] The active detection module simulates real business messages to send requests to business applications, and regularly sends the request information and response data as monitoring data to the data collection and analysis module;
[0043] The data collection and analysis module receives monitoring data sent by the passive detection module and the active detection module, and incorporates it into the business application status assessment model to obtain assessment results;
[0044] If the evaluation result is in line with expectations, the business application is considered to be running normally; otherwise, the business application is considered to be running abnormally, and the abnormal business application and corresponding monitoring data are pushed to the alarm module;
[0045] The alarm module notifies relevant operation and maintenance personnel of abnormal business applications and corresponding monitoring data, and performs business application fault self-recovery operations according to the configured business fault handling rules.
[0046] Compared with the prior art, the present invention has the following beneficial effects:
[0047] The present invention sets up an active detection module to enable a business active detection mode for full-view detection of applications. Compared with the traditional mode, which only collects indicators that are not closely related to the business to judge the application operation status and is more difficult to detect situations such as program failure, the business active detection mode of the present invention can simulate real transaction scenarios to complete detection and realize advanced business behavior prediction.
[0048] Compared with traditional evaluations that only consider indicator thresholds without associating them with instance hardware information, business transaction time, and actual business transactions, the business application status evaluation model of the present invention is more accurate.
[0049] In terms of acquiring training data, the present invention sets up a dedicated transaction information recording, which will enter the information used by the application status assessment model during the transaction into a local file as a training data source, and can comprehensively train the application status assessment model.
[0050] The present invention can proactively detect business requests after being set up, paving the way for upcoming business calls and truly realizing business tests identical to real business scenarios. The business application status evaluation parameters are diversified, and by introducing more business attributes, business monitoring no longer considers a single indicator, but instead establishes a comprehensive application status evaluation model to complete the evaluation of the application operation status. Transaction information recording collects business application operation information based on real scenarios, making the evaluation results of the business application status evaluation model more reliable. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 This is a diagram of the architecture of a full-view monitoring system for an application disclosed in an embodiment of the present invention;
[0052] Figure 2 A schematic diagram of the monitoring process of a full-view monitoring system disclosed in an embodiment of the present invention. DETAILED DESCRIPTION
[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.
[0054] The present invention is described in further detail below with reference to the accompanying drawings:
[0055] like Figure 1 As shown, the present invention provides an application of a full-view monitoring method, including: a passive detection module, an active detection module, a data collection and analysis module and an alarm module;
[0056] Passive detection module for:
[0057] Integrated in business applications, it receives and regularly sends monitoring data to the business collection and analysis module based on the settings of the data collection and analysis module, including multi-level monitoring data such as process level, service level, network level and logic level;
[0058] in,
[0059] The settings of the data collection and analysis module received by the passive detection module include:
[0060] Collect information from various business applications, including logs, metrics, and service link information;
[0061] Regular detection time and detection range for each business application.
[0062] Further,
[0063] Each business application can select the regular detection time, detection range, etc. according to the application situation.
[0064] Active detection module, used for:
[0065] Receive and simulate real business messages to send requests to business applications according to the settings of the data collection and analysis module, and regularly send the request information and response data as monitoring data to the data collection and analysis module;
[0066] in,
[0067] The active detection module is a standalone application that collects and analyzes the received data, including:
[0068] The service interface, detection interval, and detection time range of each business application's active detection.
[0069] Further,
[0070] The active detection module sends the request information, response results, time-consuming information and status code as monitoring data to the data collection and analysis module on a regular basis.
[0071] Specifically,
[0072] In the data collection and analysis interface, users configure the business interface, detection interval, and detection time range for each business application to actively detect.
[0073] After the configuration takes effect, the application active detection module will send request messages to the business application according to the settings, and feed back information such as application response and time consumption to the data collection and analysis module.
[0074] Data collection and analysis module for:
[0075] Receive monitoring data from the passive detection module and the active detection module, send the business application status assessment model, and obtain the assessment results. If the results meet expectations, the business application is considered to be running normally. Otherwise, the business application is considered to be running abnormally, and the abnormal business application and the corresponding monitoring data are pushed to the alarm module.
[0076] in,
[0077] The data collection and analysis module incorporates the monitoring data sent by the passive detection module and the active detection module into the business application status evaluation model respectively, and evaluates the operating status of the business application respectively.
[0078] Build a multi-dimensional and three-dimensional business application operation status assessment model based on the historical data of business application operation;
[0079] Based on the historical data of business application operation, a large application operation model is built, and the large application operation model is trained to obtain a business application operation status evaluation model.
[0080] On the one hand, the business application status assessment module can combine the big model and the historical business operation data to establish a business application status assessment big model. On the other hand, based on the data sent by the data collection and analysis module, it can judge whether the current business application is running normally and feed back the operation results to the data collection and analysis module.
[0081] Specifically,
[0082] Business application operation history data includes: hardware attributes, software attributes, log attributes, indicator attributes, service link attributes, and business attributes;
[0083] Hardware attributes include: CPU frequency, number, CPU instruction set, memory size, and memory frequency.
[0084] Software attributes include: operating system, JDK version, and compilation parameters.
[0085] Log attributes include: log time, log level, and log information.
[0086] Indicator attributes include: CPU usage, memory usage, request time, request response, status code, etc.
[0087] Service link attributes include: traceId, spanId, and call context tags.
[0088] Business attributes include: business name, version, runtime, business interface, business request field, and business response field.
[0089] Training large models for applications includes:
[0090] Configure the business application to enable transaction recording, and completely record the business application's historical operation data during the transaction period as training data.
[0091] When the monitoring data from the passive detection module arrives, the detection logs, indicators, and service link information are entered into the business application status assessment model for model inference. After judgment by the large model, if the judgment result is abnormal, the information will be handed over to the alarm center.
[0092] When the monitoring data of the active detection model arrives, the request message, response message, response time, status code, etc. are entered into the business application status assessment model for model inference. After judgment by the large model, if the judgment result is abnormal, the information will be handed over to the alarm center.
[0093] Alarm module, used for:
[0094] Send abnormal business applications and corresponding monitoring data information to operation and maintenance personnel;
[0095] Perform business application fault self-recovery operations according to the configured business fault handling rules.
[0096] like Figure 2 As shown, the present invention also provides an application full-view monitoring method, including:
[0097] S1. The passive detection module receives and sends monitoring data to the data collection and analysis module regularly according to the settings of the data collection and analysis module, including multi-level monitoring data such as process level, service level, network level, and logical level.
[0098] S2, the active detection module simulates real business messages to send requests to business applications, and regularly sends the request information and response data as monitoring data to the data collection and analysis module;
[0099] S3. The data collection and analysis module receives the monitoring data sent by the passive detection module and the active detection module, and incorporates it into the business application status assessment model to obtain the assessment results;
[0100] S4. If the evaluation result meets expectations, the business application is judged to be running normally; otherwise, the business application is judged to be running abnormally, and the abnormal business application and corresponding monitoring data are pushed to the alarm module;
[0101] S5. The alarm module notifies the relevant operation and maintenance personnel of the abnormal business application and the corresponding monitoring data, and performs business application fault self-recovery operations according to the configured business fault handling rules.
[0102] The transaction information recording of the present invention: collects business application operation information based on real scenarios, including but not limited to: CPU, memory, logs, service links, request messages, response messages, program time consumption, and operation status.
[0103] This invention adheres to the concept that active business detection should not affect the accuracy of real transaction data. By introducing dry-run request headers, dry-run customer information, dry-run data routing, etc., we isolate real data from dry-run data, ensuring the authenticity of test process execution and the isolation of test data. Based on the dry-run mode, it will not affect the actual transaction results, but the running process and the real scenario remain completely consistent. The authenticity of the detection process is guaranteed. To ensure the normal operation of the dry-run mode, for interfaces that do not enter the database, the interface can be directly called. For interfaces that do enter the database, we will have a data proxy layer. According to the setting of the request header, data operations will also enter the dry-run mode.
[0104] The present invention can proactively detect business interfaces in advance during the business operation trough period, predict business application response information in advance, achieve advanced business behavior prediction, and provide good data support for application preheating and application peak and valley behavior prediction.
[0105] The business application status assessment model of the present invention can be trained based on an existing open source large model to complete model parameter tuning for monitoring and operation, without any restrictions on the model itself.
[0106] The historical data of the business application status assessment model of the present invention may have some operating data adjusted according to different applications. For example, non-Java programs do not include JVM operating data.
[0107] In the present invention, the dry-run mode simulates the execution of a process without actually executing the operation to see what will happen, without causing any actual impact or changing the system state. The traceId in the service link attribute is a unique identifier used to mark the entire link process of a "request". No matter how many services, microservices, threads, or nodes a request spans, as long as they belong to the same user request (or transaction), they will share the same traceId. The spanId in the service link attribute is used to identify a specific operation unit (call, request, method, etc.). It is usually a fragment of a traceId (i.e., the entire request chain). A traceId can contain one or more spanIds.
[0108] Example:
[0109] Passive detection process:
[0110] In the data collection and analysis interface, users can configure the scheduled collection of information for each business application, including logs, indicators, and service link information. Each business application can select the regular detection time and detection range based on the application situation.
[0111] The data collection and analysis module collects logs, indicators, and service link information, and incorporates the information into the business application status assessment model. If the results meet expectations, the business application is considered to be running normally. Otherwise, the running status is considered abnormal, and the abnormal business application information is pushed to the alarm center.
[0112] Based on the information collected by the data collection and analysis module, the alarm center notifies relevant operation and maintenance personnel on the one hand, and performs application fault self-recovery operations according to the configured business fault handling rules on the other hand.
[0113] Active detection process:
[0114] In the data collection and analysis interface, users configure the business interface, detection interval, and detection time range for each business application to actively detect.
[0115] After the configuration takes effect, the application active detection module will send request messages to the business application according to the settings, and feed back information such as application response and time consumption to the data collection and analysis module.
[0116] The data collection and analysis module receives the response, time consumption and other information sent by the active detection module, and incorporates the information into the business application status assessment model. If the result meets the expectations, the business application is considered to be running normally. Otherwise, the running status is considered abnormal, and the abnormal business application information is pushed to the alarm center.
[0117] Based on the information collected by the data collection and analysis module, the alarm center notifies relevant operation and maintenance personnel on the one hand, and performs application fault self-recovery operations according to the configured business fault handling rules on the other hand.
[0118] Advantages of the present invention:
[0119] The present invention sets up an active detection module to enable a business active detection mode for full-view detection of applications. Compared with the traditional mode, which only collects indicators that are not closely related to the business to judge the application operation status and is more difficult to detect situations such as program failure, the business active detection mode of the present invention can simulate real transaction scenarios to complete detection and realize advanced business behavior prediction.
[0120] Compared with traditional evaluations that only consider indicator thresholds without associating them with instance hardware information, business transaction time, and actual business transactions, the business application status evaluation model of the present invention is more accurate.
[0121] In terms of acquiring training data, the present invention sets up a dedicated transaction information recording, which will enter the information used by the application status assessment model during the transaction into a local file as a training data source, and can comprehensively train the application status assessment model.
[0122] The present invention can proactively detect business requests after being set up, paving the way for upcoming business calls and truly realizing business tests identical to real business scenarios. The business application status evaluation parameters are diversified, and by introducing more business attributes, business monitoring no longer considers a single indicator, but instead establishes a comprehensive application status evaluation model to complete the evaluation of the application operation status. Transaction information recording collects business application operation information based on real scenarios, making the evaluation results of the business application status evaluation model more reliable.
[0123] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.
Claims
1. An applied full-view monitoring system, characterized in that: Includes: passive detection module, active detection module, data collection and analysis module and alarm module; The passive detection module is used to: Integrated in business applications, receiving and, according to the settings of the data collection and analysis module, regularly sending monitoring data to the business collection and analysis module, including process-level, service-level, network-level and logic-level monitoring data; The active detection module is used to: Receive and, based on the settings of the data collection and analysis module, simulate real business messages to send requests to business applications, and regularly send the request information and response data as monitoring data to the data collection and analysis module; The data collection and analysis module is used to: Receive the monitoring data sent by the passive detection module and the active detection module, send the business application status assessment model, obtain the assessment result, and if it meets the expectations, determine that the business application operation status is normal; otherwise, determine that the operation status is abnormal, and push the abnormal business application and the corresponding monitoring data to the alarm module; The alarm module is used to: Send abnormal business applications and corresponding monitoring data information to operation and maintenance personnel; Perform business application fault self-recovery operations according to the configured business fault handling rules.
2. The full-view monitoring system according to claim 1, characterized in that: The settings of the data collection and analysis module received by the passive detection module include: Collect information from various business applications, including logs, metrics, and service link information; Regular detection time and detection range for each business application.
3. The full-view monitoring system according to claim 1, characterized in that: The settings of the data collection and analysis module received by the active detection module include: The service interface, detection interval, and detection time range of each business application's active detection.
4. The full-view monitoring system according to claim 1, characterized in that: The active detection module sends the request information, response result, time-consuming information and status code as monitoring data to the data collection and analysis module on a regular basis.
5. The full-view monitoring system according to claim 1, characterized in that: The data collection and analysis module incorporates the monitoring data sent by the passive detection module and the active detection module into the business application status evaluation model respectively, and evaluates the running status of the business application respectively.
6. The full-view monitoring system according to claim 1, characterized in that: A multi-dimensional and three-dimensional business application operation status evaluation model is constructed based on the historical data of business application operation.
7. The full-view monitoring system according to claim 6, characterized in that: An application operation large model is constructed based on the business application operation historical data, and the application operation large model is trained to obtain a business application operation status evaluation model.
8. The full-view monitoring system according to claim 7, characterized in that: Business application operation history data includes: hardware attributes, software attributes, log attributes, indicator attributes, service link attributes, and business attributes; The hardware attributes include: CPU frequency, number, CPU instruction set, memory size, and memory frequency; The software attributes include: operating system, jdk version, and compilation parameters; The log attributes include: log time, log level, and log information; The indicator attributes include: CPU usage, memory usage, request time, request response, status code, etc. The service link attributes include: traceId, spanId, and call context tag; The service attributes include: service name, version, runtime, service interface, service request field, and service response field.
9. The full-view monitoring system according to claim 8, characterized in that: Run a large model for training the application, including: Configure the business application to enable transaction recording, and completely record the business application's historical running data during the transaction period as training data.
10. A full-view monitoring method for an application, using the full-view monitoring system for an application as claimed in any one of claims 1 to 9, characterized in that: include: The passive detection module receives and sends monitoring data to the data collection and analysis module regularly according to the settings of the data collection and analysis module, including multi-level monitoring data such as process level, service level, network level, and logical level; The active detection module simulates real business messages to send requests to business applications, and regularly sends the request information and response data as monitoring data to the data collection and analysis module; The data collection and analysis module receives monitoring data sent by the passive detection module and the active detection module, and incorporates it into the business application status assessment model to obtain assessment results; If the evaluation result is in line with expectations, the business application is considered to be running normally; otherwise, the business application is considered to be running abnormally, and the abnormal business application and corresponding monitoring data are pushed to the alarm module; The alarm module notifies relevant operation and maintenance personnel of abnormal business applications and corresponding monitoring data, and performs business application fault self-recovery operations according to the configured business fault handling rules.