Business system monitoring method and apparatus, computer device, and storage medium

Abstract

The application relates to a business system monitoring method and device, computer equipment and a storage medium, relates to the technical field of computers, and can be applied to the field of financial technology or other related fields. The method comprises the following steps: determining a critical value of a static monitoring index of a business system according to historical performance monitoring data of the business system; determining a monitoring threshold value of a dynamic monitoring index corresponding to the static monitoring index according to the critical value of the static monitoring index and an alarm value; and monitoring the business system according to the alarm value and the monitoring threshold value. The method can improve the accuracy of business system monitoring and ensure the accuracy of judging the health state of the business system.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a business system monitoring method, apparatus, computer equipment, and storage medium, which can be used in the financial technology field or other related fields. Background Technology

[0002] Performance testing is an essential stage in the operation and maintenance of business systems. During the performance testing phase, various indicators of the business system need to be monitored, such as throughput, concurrency, and memory usage, to measure the health status of the business system.

[0003] The existing methods for monitoring business systems mainly involve setting alarm values ​​for monitoring indicators and then monitoring the business system based on these alarm values ​​to measure its health status.

[0004] However, the above-mentioned business system monitoring methods rely solely on alarm values, which is a simplistic approach and can lead to low accuracy in monitoring business systems, thereby reducing the accuracy of assessing the health status of the business systems. Summary of the Invention

[0005] Therefore, it is necessary to provide a business system monitoring method, apparatus, computer equipment, and storage medium that can improve the accuracy of business system monitoring and thus ensure the accuracy of judging the health status of the business system, in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a business system monitoring method, which includes:

[0007] Based on the historical performance monitoring data of the business system, determine the critical values ​​of the static monitoring indicators of the business system;

[0008] Based on the critical values ​​and alarm values ​​of the static monitoring indicators, determine the monitoring thresholds of the corresponding dynamic monitoring indicators.

[0009] The business system is monitored based on alarm values ​​and monitoring thresholds.

[0010] In one embodiment, the critical values ​​of static monitoring indicators of the business system are determined based on historical performance monitoring data of the business system, including:

[0011] Based on the historical performance monitoring data of the business system, construct the long-tail distribution curve of the static monitoring indicators of the business system;

[0012] Based on the long-tail distribution curve, determine the critical values ​​of static monitoring indicators.

[0013] In one embodiment, the critical value of the static monitoring indicator is determined based on the long-tail distribution curve, including:

[0014] The abrupt change point of the long-tailed distribution curve is determined by the slope of adjacent points in the long-tailed distribution curve.

[0015] The value of the static monitoring indicator corresponding to the mutation point is used as the critical value of the static monitoring indicator.

[0016] In one embodiment, the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator is determined based on the threshold value and alarm value of the static monitoring indicator, including:

[0017] Determine the difference between the critical value and the alarm value of the static monitoring indicator;

[0018] The ratio between the difference and the collection period of the dynamic monitoring indicator corresponding to the static monitoring indicator is used as the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator.

[0019] In one embodiment, the method further includes:

[0020] Based on the processing capacity of the business system, determine the first threshold of the static monitoring indicators of the business system.

[0021] Based on the system stability of the business system, determine the second threshold of the static monitoring indicators of the business system;

[0022] Based on the first threshold and the second threshold, determine the alarm values ​​of the static monitoring indicators of the business system.

[0023] In one embodiment, the method further includes:

[0024] If the actual value of a static monitoring indicator is found to be greater than the alarm value, and / or the actual value of a dynamic monitoring indicator is found to be greater than the monitoring threshold, then the business system is determined to be abnormal.

[0025] Obtain log data when the business system encounters an anomaly, and then feed the log data back to the operations and maintenance terminal.

[0026] In one embodiment, the method further includes:

[0027] If the actual value of a static monitoring indicator is found to be greater than the alarm value, the alarm level shall be determined according to the type of static monitoring indicator.

[0028] Alarms should be handled according to their alarm levels.

[0029] Secondly, this application also provides a business system monitoring device, which includes:

[0030] The threshold value determination module is used to determine the threshold values ​​of static monitoring indicators of the business system based on the historical performance monitoring data of the business system.

[0031] The threshold determination module is used to determine the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator based on the critical value and alarm value of the static monitoring indicator.

[0032] The system monitoring module is used to monitor the business system based on alarm values ​​and monitoring thresholds.

[0033] Thirdly, this application also provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0034] Based on the historical performance monitoring data of the business system, determine the critical values ​​of the static monitoring indicators of the business system;

[0035] Based on the critical values ​​and alarm values ​​of the static monitoring indicators, determine the monitoring thresholds of the corresponding dynamic monitoring indicators.

[0036] The business system is monitored based on alarm values ​​and monitoring thresholds.

[0037] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0038] Based on the historical performance monitoring data of the business system, determine the critical values ​​of the static monitoring indicators of the business system;

[0039] Based on the critical values ​​and alarm values ​​of the static monitoring indicators, determine the monitoring thresholds of the corresponding dynamic monitoring indicators.

[0040] The business system is monitored based on alarm values ​​and monitoring thresholds.

[0041] Fifthly, this application also provides a computer program product, which includes a computer program that, when executed by a processor, performs the following steps:

[0042] Based on the historical performance monitoring data of the business system, determine the critical values ​​of the static monitoring indicators of the business system;

[0043] Based on the critical values ​​and alarm values ​​of the static monitoring indicators, determine the monitoring thresholds of the corresponding dynamic monitoring indicators.

[0044] The business system is monitored based on alarm values ​​and monitoring thresholds.

[0045] The aforementioned business system monitoring method, apparatus, computer equipment, and storage medium determine the critical values ​​of static monitoring indicators for the business system based on historical performance monitoring data. Then, based on these critical values ​​and the alarm values ​​of the static monitoring indicators, they determine the monitoring thresholds for the corresponding dynamic monitoring indicators. Furthermore, the business system is monitored based on the alarm values ​​of the static monitoring indicators and the corresponding monitoring thresholds for the dynamic monitoring indicators. This approach, by introducing monitoring thresholds for dynamic monitoring indicators corresponding to static monitoring indicators and monitoring the business system based on these thresholds and alarm values, enriches the monitoring data compared to related technologies that rely solely on alarm values. This results in more comprehensive monitoring, improves the accuracy of business system monitoring, and ultimately ensures the accuracy of assessing the health status of the business system. Attached Figure Description

[0046] Figure 1 This is an application environment diagram of a business system monitoring method in one embodiment;

[0047] Figure 2 This is a flowchart illustrating a business system monitoring method in one embodiment;

[0048] Figure 3 This is a flowchart illustrating the process of determining the critical value of a static monitoring indicator in one embodiment.

[0049] Figure 4 This is a flowchart illustrating the process of determining the monitoring threshold of a dynamic monitoring indicator in one embodiment.

[0050] Figure 5 This is a flowchart illustrating the business system monitoring method in another embodiment;

[0051] Figure 6 This is a structural block diagram of a business system monitoring device in one embodiment;

[0052] Figure 7 This is a structural block diagram of the business system monitoring device in another embodiment;

[0053] Figure 8 This is a structural block diagram of the business system monitoring device in another embodiment;

[0054] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0056] The business system monitoring method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, the maintenance terminal 102 communicates with the server 104 via a network. The maintenance terminal is the terminal used for maintaining the business system. The data storage system stores the data that the server 104 needs to process. The data storage system can be integrated onto the server 104 or placed on the cloud or other network servers. During the monitoring of the business system, the server 104 determines the critical values ​​of the static monitoring indicators of the business system based on historical performance monitoring data; based on the critical values ​​of the static monitoring indicators and the alarm values ​​of the static monitoring indicators, it determines the monitoring thresholds of the dynamic monitoring indicators corresponding to the static monitoring indicators; based on the alarm values ​​and monitoring thresholds, it monitors the business system; furthermore, the server 104 interacts with the maintenance terminal 102 via the network, feeding back the log data obtained when the business system is abnormal to the maintenance terminal 102, so that the maintenance terminal 102 can promptly repair the business system based on the obtained log data.

[0057] The maintenance terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0058] In the field of banking business system monitoring, monitoring various indicators within the business system is an essential step. Current business system monitoring methods primarily involve setting alarm values ​​for monitoring indicators and then monitoring the business system based on these alarm values ​​to measure its health status.

[0059] However, existing business system monitoring methods rely solely on alarm values, which is a simplistic approach and can lead to low monitoring accuracy, failing to guarantee the accuracy of assessing the health status of the business system.

[0060] Based on this, in one embodiment, such as Figure 2 As shown, a business system monitoring method is provided, which can be applied to... Figure 1 Taking server 104 as an example, the following steps are included:

[0061] S201, Based on the historical performance monitoring data of the business system, determine the critical values ​​of the static monitoring indicators of the business system.

[0062] In this embodiment, historical performance monitoring data refers to performance data monitored within historical time periods prior to the current time. Static monitoring metrics are those configured by the operations and maintenance team that are highly relevant to the business system. These metrics can be used to monitor the business system at a single moment or a single collection period, such as memory usage, CPU utilization, and interface response time. Optionally, in this embodiment, multiple static monitoring metrics can be used simultaneously to monitor the business system.

[0063] The critical value of a static monitoring indicator is its boundary value; specifically, it's the value at which a static monitoring indicator experiences a significant change. For example, when analyzing historical performance monitoring data, if the analysis shows that the actual value of a certain static monitoring indicator suddenly changes drastically at a certain moment, and the trend of the indicator's actual value is relatively flat before and after that moment, then the actual value of the static monitoring indicator at the moment of the significant change can be taken as the critical value of that static monitoring indicator. Optionally, different static monitoring indicators may have different critical values.

[0064] Specifically, the server analyzes historical performance monitoring data of the business system to obtain the critical value corresponding to each static monitoring indicator of the business system. For example, one implementation is to input the acquired historical performance monitoring data into a pre-trained neural network model. The model, based on its own parameters, comprehensively analyzes the actual values ​​of each static monitoring indicator captured from the historical performance monitoring data and outputs the critical value corresponding to each static monitoring indicator of the business system. Another implementation is to extract relevant data for each static monitoring indicator from the acquired historical performance monitoring data and perform statistical analysis on the extracted relevant data to obtain the actual value of the static monitoring indicator at each historical time. Based on the actual value of the static monitoring indicator at each historical time, the change range of the static monitoring indicator is analyzed to obtain the actual value corresponding to the large change range of the static monitoring indicator. Furthermore, the actual value corresponding to the large change range of the static monitoring indicator is used as the critical value corresponding to the static monitoring indicator.

[0065] S202, Based on the critical values ​​and alarm values ​​of the static monitoring indicators, determine the monitoring thresholds of the dynamic monitoring indicators corresponding to the static monitoring indicators.

[0066] In this embodiment, the alarm value of the static monitoring indicator can be the alarm threshold of the static monitoring indicator preset by the operation and maintenance party according to the actual situation of the business system; or it can be the alarm value of the static monitoring indicator of the business system determined according to the processing capacity and system stability of the business system.

[0067] Optionally, a first threshold for the static monitoring indicators of the business system can be determined based on the processing capacity of the business system; a second threshold for the static monitoring indicators of the business system can be determined based on the system stability of the business system; and alarm values ​​for the static monitoring indicators of the business system can be determined based on the first and second thresholds.

[0068] Among them, the processing capacity of the business system is the maximum processing capacity of the business system at present; the system stability of the business system is the current operational stability of the business system.

[0069] Specifically, the processing capacity of a business system can be determined based on its data volume, memory usage, etc. Then, based on the determined processing capacity, a first threshold for the static monitoring indicators can be determined according to the pre-defined correlation between the processing capacity and the threshold of the static monitoring indicators. Optionally, the larger the processing capacity of the business system, the higher the first threshold of the static monitoring indicators.

[0070] Furthermore, the system stability of the business system is determined based on the interruptions and crashes that occurred during the business processing in the historical period. Based on the determined stability of the business system, a second threshold for the static monitoring indicators of the business system is determined. Optionally, the higher the system stability of the business system, the higher the second threshold for the static monitoring indicators.

[0071] Then, based on the determined first and second thresholds, and according to pre-set logic, the alarm values ​​of the static monitoring indicators of the business system are determined. For example, the minimum value between the first and second thresholds, or the average value between the first and second thresholds, can be used as the alarm values ​​of the static monitoring indicators of the business system.

[0072] For example, for the static monitoring metric of CPU utilization, if the business system is analyzed and it is determined that the business system can handle a large amount of business, then the first threshold for the CPU utilization of the business system is set to 80%. Furthermore, if it is determined that the stability of the business system is poor, then the second threshold for the CPU utilization of the business system can be set to 50%. Then, the average of the first threshold and the second threshold, i.e., 65%, is taken as the alarm value for the CPU utilization of the business system.

[0073] Understandably, setting different alarm values ​​for the static monitoring indicators of a business system based on its processing capacity and system stability improves the accuracy of monitoring the business system, thereby improving the accuracy of judging the monitoring status of the business system.

[0074] Optionally, in this embodiment, the alarm values ​​for different static monitoring indicators are different. Furthermore, in this embodiment, for any static monitoring indicator, if the value of that indicator is detected to be greater than the alarm value, an alarm is triggered.

[0075] In contrast to static monitoring metrics, dynamic monitoring metrics are those configured by the operations and maintenance team to monitor the business system at different time periods or collection cycles. Examples include memory usage change rate, CPU usage change rate, and interface response time change rate. Optionally, in this embodiment, multiple dynamic monitoring metrics can be used simultaneously to monitor the business system. The monitoring threshold for a dynamic monitoring metric is the threshold set for that metric. Optionally, in this embodiment, if the value of a dynamic monitoring metric is detected to be greater than the monitoring threshold, it is determined that an anomaly has occurred in the business system.

[0076] Specifically, after determining the threshold values ​​of static monitoring indicators, and combining them with pre-set alarm values ​​for these indicators, a corresponding monitoring threshold is set for each dynamic monitoring indicator corresponding to each static monitoring indicator, based on pre-defined calculation logic. For example, one implementation involves inputting the determined threshold values ​​and alarm values ​​of the static monitoring indicators into a pre-trained neural network model. This model then analyzes and calculates the threshold values ​​and alarm values ​​of the static monitoring indicators, outputting the monitoring threshold values ​​for the corresponding dynamic monitoring indicators.

[0077] S203 monitors the business system based on alarm values ​​and monitoring thresholds.

[0078] Specifically, after determining the monitoring thresholds for the dynamic monitoring indicators of the business system, the static and dynamic monitoring indicators of the business system are monitored in combination with the alarm values ​​of the static monitoring indicators of the business system, thereby realizing the monitoring of the business system.

[0079] The aforementioned business system monitoring method determines the critical values ​​of static monitoring indicators for the business system based on historical performance monitoring data. Then, based on these critical values ​​and the alarm values ​​of the static monitoring indicators, it determines the monitoring thresholds for the corresponding dynamic monitoring indicators. Further, it monitors the business system based on both the alarm values ​​and the corresponding dynamic monitoring thresholds. This approach, by introducing monitoring thresholds for dynamic monitoring indicators corresponding to static indicators and monitoring the business system based on these thresholds and alarm values, enriches the monitoring data compared to related technologies that rely solely on alarm values. This results in more comprehensive monitoring, improves the accuracy of business system monitoring, and ultimately ensures the accuracy of assessing the health status of the business system.

[0080] Based on the above embodiments, as an optional approach in this application embodiment, after monitoring the business system according to the alarm value and the monitoring threshold, if the actual value of the static monitoring indicator is found to be greater than the alarm value, and / or the actual value of the dynamic monitoring indicator is greater than the monitoring threshold, then the business system is determined to be abnormal; log data when the business system is abnormal is obtained, and the log data is fed back to the operation and maintenance terminal.

[0081] In this embodiment, the actual value of any static monitoring indicator at any time or in any collection period can be obtained directly through the monitoring business system; the dynamic monitoring indicator corresponding to the static monitoring indicator can be calculated and determined based on the actual value of the static monitoring indicator at different times or in different collection periods.

[0082] For example, for the dynamic monitoring metric of interface response time change rate, the interface response time of each collection period in the historical performance monitoring data of the business system is captured, and the interface response time of each collection period is analyzed to obtain the average response time of a certain interface within a day. If the average response time of the interface in the previous acquisition cycle is t1 and the average response time of the interface in the current acquisition cycle is t2, the rate of change of the response time v of the interface can be calculated by formula (1).

[0083]

[0084] Here, μ is the sensitivity factor, ranging from 0.5 to 1. The smaller the value, the more sensitive it is to changes in interface response time. For example, assuming a sensitivity factor μ is 0.5, if the average response time of the interface over a day... The average response time t1 of the interface is 98ms in the previous acquisition cycle and 255ms in the current acquisition cycle. Therefore, the response time change rate v of the interface is 78.5% as calculated by formula (1).

[0085] Specifically, static and dynamic monitoring metrics of the business system are monitored. The actual values ​​of the monitored static metrics are compared with their alarm values, and the actual values ​​of the monitored dynamic metrics are compared with their monitoring thresholds. If the actual value of a static monitoring metric is greater than its alarm value, and / or the actual value of a dynamic monitoring metric is greater than its monitoring threshold, then an anomaly can be determined in the business system.

[0086] For example, if the monitoring threshold for the interface response time change rate is determined to be 30%, and the obtained interface response time change rate of 78.5% is compared with the monitoring threshold for the interface response time change rate, it can be seen that the interface response time change rate is much greater than the monitoring threshold for the interface response time change rate, and thus it can be determined that an anomaly has occurred in the business system.

[0087] Furthermore, log data corresponding to the time when static monitoring indicators show abnormalities, and / or log data corresponding to a period of time when dynamic monitoring indicators show abnormalities, are obtained. Then, the obtained log data is fed back to the operation and maintenance terminal through the network, so that the operation and maintenance party can analyze the log data with abnormal monitoring data through the operation and maintenance terminal, and then recover from the faults that occur in the business system.

[0088] It's important to note that with the emergence of large-scale applications, system logs are becoming increasingly massive. Current technology involves feeding all log data generated by the business system back to the operations and maintenance (O&M) team for monitoring. However, the sheer volume of log data generated by business systems makes it impossible to guarantee that O&M teams can collect log data in a timely manner when anomalies occur, thus hindering their ability to recover from failures.

[0089] In this embodiment, by monitoring both static and dynamic monitoring indicators of the business system, if the actual value of a static monitoring indicator exceeds the alarm value, and / or the actual value of a dynamic monitoring indicator exceeds the monitoring threshold, an anomaly is determined to have occurred in the business system. Log data of the business system at the time of the anomaly is collected and fed back to the maintenance terminal via the network, allowing the maintenance team to analyze the abnormal log data and restore the health of the business system. This improves the accuracy of business system monitoring and enables timely feedback of abnormal logs to the maintenance team, facilitating timely recovery from system failures and ensuring the health of the business system.

[0090] Furthermore, in order to conduct timely and targeted inspections of the business system, after monitoring the business system based on alarm values ​​and monitoring thresholds, if the actual value of the static monitoring indicator is found to be greater than the alarm value, the alarm level is determined according to the type of static monitoring indicator; and alarm processing is carried out according to the alarm level.

[0091] It should be noted that when there are multiple static monitoring metrics, they can be categorized into several types based on their impact on the business system. For example, static monitoring metrics such as CPU utilization and memory utilization can be set as "important," while metrics such as interface response time and concurrency can be set as "normal." Furthermore, the alarm level for static monitoring metrics set to "important" can be set to "high," and the alarm level for static monitoring metrics set to "normal" can be set to "low."

[0092] Specifically, for each static monitoring indicator, if the actual value of the static monitoring indicator is found to be greater than the alarm value, the type of the static monitoring indicator is determined, and then the alarm level corresponding to the static monitoring indicator is determined. Furthermore, according to the alarm level of the static monitoring indicator, the corresponding alarm processing is carried out so that the operation and maintenance party can quickly conduct targeted inspections of the business system.

[0093] For example, if the actual CPU utilization value at the current moment is detected to be greater than the alarm value, the type of CPU utilization is determined, classifying it as a critical type, and thus the corresponding alarm level is set to high. Furthermore, based on the high alarm level, the server can perform corresponding high-level alarm processing. For instance, the server can control indicator lights to flash rapidly and issue an alarm via an audio module to notify the operations and maintenance team that a critical type of static monitoring indicator in the business system has an anomaly, and instruct the operations and maintenance team to handle the anomaly in the business system.

[0094] For example, if the actual value of the interface response time at the current moment is detected to be greater than the alarm value, the type of the interface response time is determined, and it is identified as a normal type, thus the alarm level is determined to be low. Furthermore, based on the low alarm level, the server can perform corresponding low-level alarm handling. For instance, the server can control an indicator light to flash slowly to notify the operations and maintenance team that a normal type of static monitoring indicator in the business system has become abnormal, and instruct the operations and maintenance team to handle the abnormal problem in the business system.

[0095] Understandably, by setting different types for static monitoring metrics, when a static monitoring metric exceeds the alarm value, the appropriate alarm handling is applied based on the different alarm levels corresponding to the metric type, thus improving the flexibility of the solution. Furthermore, introducing alarm levels allows for the instruction of operations and maintenance personnel to prioritize handling higher-alarm-level anomalies in the business system when multiple static monitoring metrics simultaneously exceed their alarm values. This means that after resolving higher-alarm-level anomalies, lower-alarm-level anomalies are addressed, ensuring the health of the business system.

[0096] To improve the accuracy of determining the critical values ​​of static monitoring indicators, in one embodiment, such as Figure 3 As shown, the above S201 is further refined. Specifically, it may include the following steps:

[0097] S301, based on the historical performance monitoring data of the business system, construct the long-tail distribution curve of the static monitoring indicators of the business system.

[0098] In this embodiment, the long-tail distribution curve is a distribution curve drawn based on the long-tail effect using historical performance monitoring data of the business system. The long-tail effect is commonly applied in the economic field. In the economic field, from the perspective of people's needs, most demand is concentrated at the head, which we can call the popular demand, while the demand distributed at the tail is personalized, scattered, and small in quantity. This differentiated, small amount of demand forms a long "tail" on the demand curve, and the so-called long-tail effect lies in its quantity; adding up all the non-popular markets creates a market larger than the popular market. Take books as an example: statistics show that a physical bookstore has an average of 130,000 titles on its shelves. However, more than half of an e-commerce company's sales come from books ranked outside the top 130,000 on its bestseller list. Based on the e-commerce company's statistics, this means that the market formed by books not sold in general bookstores is larger than that of books displayed on bookstore shelves. Furthermore, the long-tail effect is also applicable to the data analysis of static monitoring indicators of the business system.

[0099] Specifically, the server acquires historical performance monitoring data of the business system. Based on this data, it extracts the actual values ​​of static monitoring indicators for each time point or collection period. Furthermore, it analyzes these static monitoring indicator values ​​to construct a long-tail distribution curve for the business system's static monitoring indicators. For example, by capturing and analyzing the CPU utilization at each time point in the historical performance monitoring data, it can be seen that the CPU utilization of the business system exhibits a long-tail effect over time. Therefore, based on the CPU utilization within a day from the historical performance monitoring data, the distribution of the business system's CPU utilization over time can be plotted as a curve, which serves as the long-tail distribution curve of CPU utilization.

[0100] S302. Based on the long-tail distribution curve, determine the critical values ​​of static monitoring indicators.

[0101] Specifically, after constructing the long-tail distribution curve of the static monitoring indicator, the constructed long-tail distribution curve can be input into a pre-defined neural network model, which will then analyze the long-tail distribution curve to determine the critical value of the static monitoring indicator.

[0102] Optionally, the abrupt change point of the long-tailed distribution curve can be determined based on the slope of adjacent points in the curve; the value of the static monitoring indicator corresponding to the abrupt change point can be used as the critical value of the static monitoring indicator.

[0103] Specifically, after constructing the long-tail distribution curve of the static monitoring indicator, the slope between each pair of adjacent points in the long-tail distribution curve can be calculated. The calculated slopes between each pair of adjacent points are compared, and the two adjacent points with the largest slope are determined. Either of these two points is taken as the abrupt change point of the long-tail distribution curve. Furthermore, the value of the static monitoring indicator corresponding to the determined abrupt change point is taken as the critical value of the static monitoring indicator. For example, for the long-tail distribution curve corresponding to the static monitoring indicator of CPU utilization, where the horizontal axis represents CPU utilization and the vertical axis represents time percentage, the slope between each pair of points on the long-tail distribution curve is calculated, and the point with the largest slope is determined by comparison. This point is taken as the abrupt change point, and the 33% value corresponding to the abrupt change point is determined as the critical value of CPU utilization.

[0104] For example, by capturing and analyzing the interface response times for each collection period in historical performance monitoring data, it can be seen that the interface response time of the business system also exhibits a long-tail effect relative to the number of requests. Further analysis of historical interface response time monitoring data reveals that the average interface response time of the business system is 120 milliseconds. When the interface response time exceeds 92 milliseconds, the number of requests decreases significantly. Therefore, 92 milliseconds can be considered as the critical value for the interface response time of the business system.

[0105] Understandably, by analyzing historical performance monitoring data, a long-tail distribution curve of the static monitoring indicators of the business system is constructed. Furthermore, based on the constructed long-tail distribution curve, the critical values ​​of the static monitoring indicators are determined, which improves the accuracy of determining the critical values ​​of the static monitoring indicators, and thus improves the accuracy of monitoring the business system.

[0106] To improve the accuracy of determining the monitoring thresholds for dynamic monitoring indicators, in one embodiment, such as Figure 4 As shown, the above S202 is further refined. Specifically, it may include the following steps:

[0107] S401, determine the difference between the critical value and the alarm value of the static monitoring indicator.

[0108] Specifically, for each static monitoring indicator, after determining the threshold value and alarm value of the static monitoring indicator, the threshold value and alarm value of the static monitoring indicator are subtracted to obtain the difference between the threshold value and alarm value of the static monitoring indicator.

[0109] For example, if the critical value of CPU utilization of a business system is determined to be 33% and the alarm value is 80%, then the difference between the critical value and the alarm value of CPU utilization of the business system can be determined to be 80% - 33% = 47%.

[0110] S402, the ratio between the difference and the collection period of the dynamic monitoring indicator corresponding to the static monitoring indicator is used as the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator.

[0111] In this embodiment, the collection period of the dynamic monitoring indicators corresponding to the static monitoring indicators is the pre-set time period for collecting changes in the static monitoring indicators.

[0112] Specifically, for each static monitoring indicator, after determining the difference between the threshold and alarm value of the static monitoring indicator, it is compared with the pre-set collection period of the dynamic monitoring indicator corresponding to the static monitoring indicator to obtain the ratio between the difference and the collection period of the dynamic monitoring indicator corresponding to the static monitoring indicator, and the ratio is used as the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator.

[0113] For example, if the difference between the critical value and the alarm value of the CPU utilization rate of the business system is determined to be 47%, and the preset collection period of CPU change rate is 20s, then the monitoring threshold of CPU change rate can be determined to be 47% / 20s = 2.35%.

[0114] It is understood that in this embodiment, after determining the difference between the threshold value and the alarm value of the static monitoring indicator, the difference is compared with the collection period of the dynamic monitoring indicator corresponding to the static monitoring indicator, and the ratio is used as the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator. This provides an optional method for accurately determining the monitoring threshold of the dynamic monitoring indicator, thereby laying the foundation for improving the accuracy of business system monitoring and ensuring the accuracy of judging the health status of the business system.

[0115] In one embodiment, such as Figure 5 As shown, an optional example of a business system monitoring method is provided.

[0116] The specific process is as follows:

[0117] S501, based on the historical performance monitoring data of the business system, constructs the long-tail distribution curve of the static monitoring indicators of the business system.

[0118] S502, determine the abrupt change point of the long-tailed distribution curve based on the slope of adjacent points in the long-tailed distribution curve.

[0119] S503, the value of the static monitoring indicator corresponding to the mutation point, is used as the critical value of the static monitoring indicator.

[0120] S504, based on the processing capacity of the business system, determine the first threshold of the static monitoring indicators of the business system.

[0121] S505, based on the system stability of the business system, determines the second threshold of the static monitoring indicators of the business system.

[0122] S506, Based on the first threshold and the second threshold, determine the alarm value of the static monitoring indicators of the business system.

[0123] S507 determines the difference between the critical value and the alarm value of the static monitoring indicator.

[0124] S508 uses the ratio between the difference and the collection period of the dynamic monitoring indicator corresponding to the static monitoring indicator as the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator.

[0125] S509 monitors the business system based on alarm values ​​and monitoring thresholds.

[0126] S510 If the actual value of a static monitoring indicator is found to be greater than the alarm value, and / or the actual value of a dynamic monitoring indicator is found to be greater than the monitoring threshold, then the business system is determined to be abnormal.

[0127] S511 retrieves log data when the business system encounters an anomaly and sends the log data back to the operations and maintenance terminal.

[0128] Furthermore, if the actual value of a static monitoring indicator is found to be greater than the alarm value, the alarm level can be determined based on the type of static monitoring indicator; and alarm processing can be carried out based on the alarm level.

[0129] The specific processes of S501-S511 described above can be found in the description of the above method embodiments. Their implementation principles and technical effects are similar, and will not be repeated here.

[0130] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0131] Based on the same inventive concept, this application also provides a business system monitoring device for implementing the business system monitoring method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the business system monitoring device provided below can be found in the limitations of the business system monitoring method described above, and will not be repeated here.

[0132] In one embodiment, such as Figure 6 As shown, a business system monitoring device 1 is provided, including: a threshold value determination module 10, a threshold determination module 20, and a system monitoring module 30, wherein:

[0133] The threshold value determination module 10 is used to determine the threshold values ​​of the static monitoring indicators of the business system based on the historical performance monitoring data of the business system.

[0134] The threshold determination module 20 is used to determine the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator based on the critical value and alarm value of the static monitoring indicator.

[0135] The system monitoring module 30 is used to monitor the business system based on alarm values ​​and monitoring thresholds.

[0136] In one embodiment, in the above Figure 6 On the basis of, such as Figure 7 As shown, the above-mentioned critical value determination module 10 may include:

[0137] Curve construction unit 11 is used to construct the long-tail distribution curve of the static monitoring indicators of the business system based on the historical performance monitoring data of the business system.

[0138] The critical value determination unit 12 is used to determine the critical value of the static monitoring indicator based on the long-tail distribution curve.

[0139] In one embodiment, the threshold determination unit 12 described above can also be used for:

[0140] Based on the slope of adjacent points in the long-tail distribution curve, the abrupt change point of the long-tail distribution curve is determined; the value of the static monitoring indicator corresponding to the abrupt change point is taken as the critical value of the static monitoring indicator.

[0141] In one embodiment, in the above Figure 6 or Figure 7 On the basis of, such as Figure 8 As shown, the threshold determination module 20 described above may include:

[0142] The difference determination unit 21 is used to determine the difference between the critical value and the alarm value of the static monitoring indicator;

[0143] The threshold determination unit 22 is used to take the ratio between the difference and the collection period of the dynamic monitoring indicator corresponding to the static monitoring indicator as the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator.

[0144] In one embodiment, the business system monitoring device may further include:

[0145] The alarm value determination module is used to determine the first threshold of the static monitoring indicators of the business system based on the processing capacity of the business system; determine the second threshold of the static monitoring indicators of the business system based on the system stability of the business system; and determine the alarm value of the static monitoring indicators of the business system based on the first threshold and the second threshold.

[0146] In one embodiment, the business system monitoring device may further include a log feedback module, which is specifically used for:

[0147] If the actual value of a static monitoring indicator is found to be greater than the alarm value, and / or the actual value of a dynamic monitoring indicator is found to be greater than the monitoring threshold, then the business system is determined to be abnormal; the log data when the business system is abnormal is obtained and the log data is fed back to the operation and maintenance terminal.

[0148] In one embodiment, the business system monitoring device may further include an alarm module, which is specifically used for:

[0149] If the actual value of a static monitoring indicator is found to be greater than the alarm value, the alarm level is determined according to the type of static monitoring indicator; and alarm processing is carried out according to the alarm level.

[0150] Each module in the aforementioned business system monitoring device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0151] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores historical performance monitoring data, critical values ​​of static monitoring indicators, and alarm thresholds. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a business system monitoring method.

[0152] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0153] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0154] Based on the historical performance monitoring data of the business system, determine the critical values ​​of the static monitoring indicators of the business system;

[0155] Based on the critical values ​​and alarm values ​​of the static monitoring indicators, determine the monitoring thresholds of the corresponding dynamic monitoring indicators.

[0156] The business system is monitored based on alarm values ​​and monitoring thresholds.

[0157] In one embodiment, when the processor executes the logic of a computer program to determine the critical values ​​of static monitoring indicators of the business system based on historical performance monitoring data of the business system, it also implements the following steps:

[0158] Based on the historical performance monitoring data of the business system, construct the long-tail distribution curve of the static monitoring indicators of the business system;

[0159] Based on the long-tail distribution curve, determine the critical values ​​of static monitoring indicators.

[0160] In one embodiment, when the processor executes the logic of the computer program to determine the critical value of the static monitoring indicator based on the long-tail distribution curve, it also implements the following steps:

[0161] The abrupt change point of the long-tailed distribution curve is determined by the slope of adjacent points in the long-tailed distribution curve.

[0162] The value of the static monitoring indicator corresponding to the mutation point is used as the critical value of the static monitoring indicator.

[0163] In one embodiment, when the processor executes the logic of a computer program to determine the monitoring threshold of a dynamic monitoring indicator corresponding to a static monitoring indicator based on the threshold and alarm value of the static monitoring indicator, it also implements the following steps:

[0164] Determine the difference between the critical value and the alarm value of the static monitoring indicator;

[0165] The ratio between the difference and the collection period of the dynamic monitoring indicator corresponding to the static monitoring indicator is used as the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator.

[0166] In one embodiment, when the processor executes a computer program, it also performs the following steps:

[0167] Based on the processing capacity of the business system, determine the first threshold of the static monitoring indicators of the business system.

[0168] Based on the system stability of the business system, determine the second threshold of the static monitoring indicators of the business system;

[0169] Based on the first threshold and the second threshold, determine the alarm values ​​of the static monitoring indicators of the business system.

[0170] In one embodiment, when the processor executes a computer program, it also performs the following steps:

[0171] If the actual value of a static monitoring indicator is found to be greater than the alarm value, and / or the actual value of a dynamic monitoring indicator is found to be greater than the monitoring threshold, then the business system is determined to be abnormal.

[0172] Obtain log data when the business system encounters an anomaly, and then feed the log data back to the operations and maintenance terminal.

[0173] In one embodiment, when the processor executes a computer program, it also performs the following steps:

[0174] If the actual value of a static monitoring indicator is found to be greater than the alarm value, the alarm level shall be determined according to the type of static monitoring indicator.

[0175] Alarms should be handled according to their alarm levels.

[0176] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0177] Based on the historical performance monitoring data of the business system, determine the critical values ​​of the static monitoring indicators of the business system;

[0178] Based on the critical values ​​and alarm values ​​of the static monitoring indicators, determine the monitoring thresholds of the corresponding dynamic monitoring indicators.

[0179] The business system is monitored based on alarm values ​​and monitoring thresholds.

[0180] In one embodiment, when the logic of the computer program determining the critical values ​​of the static monitoring indicators of the business system based on historical performance monitoring data of the business system is executed by the processor, the following steps are also implemented:

[0181] Based on the historical performance monitoring data of the business system, construct the long-tail distribution curve of the static monitoring indicators of the business system;

[0182] Based on the long-tail distribution curve, determine the critical values ​​of static monitoring indicators.

[0183] In one embodiment, when the logic of the computer program determining the critical value of the static monitoring indicator based on the long-tail distribution curve is executed by the processor, the following steps are also implemented:

[0184] The abrupt change point of the long-tailed distribution curve is determined by the slope of adjacent points in the long-tailed distribution curve.

[0185] The value of the static monitoring indicator corresponding to the mutation point is used as the critical value of the static monitoring indicator.

[0186] In one embodiment, when the logic of the computer program determining the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator based on the threshold and alarm value of the static monitoring indicator is executed by the processor, the following steps are also implemented:

[0187] Determine the difference between the critical value and the alarm value of the static monitoring indicator;

[0188] The ratio between the difference and the collection period of the dynamic monitoring indicator corresponding to the static monitoring indicator is used as the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator.

[0189] In one embodiment, when the computer program is executed by the processor, it also performs the following steps:

[0190] Based on the processing capacity of the business system, determine the first threshold of the static monitoring indicators of the business system.

[0191] Based on the system stability of the business system, determine the second threshold of the static monitoring indicators of the business system;

[0192] Based on the first threshold and the second threshold, determine the alarm values ​​of the static monitoring indicators of the business system.

[0193] In one embodiment, when the computer program is executed by the processor, it also performs the following steps:

[0194] If the actual value of a static monitoring indicator is found to be greater than the alarm value, and / or the actual value of a dynamic monitoring indicator is found to be greater than the monitoring threshold, then the business system is determined to be abnormal.

[0195] Obtain log data when the business system encounters an anomaly, and then feed the log data back to the operations and maintenance terminal.

[0196] In one embodiment, when the computer program is executed by the processor, it also performs the following steps:

[0197] If the actual value of a static monitoring indicator is found to be greater than the alarm value, the alarm level shall be determined according to the type of static monitoring indicator.

[0198] Alarms should be handled according to their alarm levels.

[0199] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0200] Based on the historical performance monitoring data of the business system, determine the critical values ​​of the static monitoring indicators of the business system;

[0201] Based on the critical values ​​and alarm values ​​of the static monitoring indicators, determine the monitoring thresholds of the corresponding dynamic monitoring indicators.

[0202] The business system is monitored based on alarm values ​​and monitoring thresholds.

[0203] In one embodiment, when the logic of the computer program determining the critical values ​​of the static monitoring indicators of the business system based on historical performance monitoring data of the business system is executed by the processor, the following steps are also implemented:

[0204] Based on the historical performance monitoring data of the business system, construct the long-tail distribution curve of the static monitoring indicators of the business system;

[0205] Based on the long-tail distribution curve, determine the critical values ​​of static monitoring indicators.

[0206] In one embodiment, when the logic of the computer program determining the critical value of the static monitoring indicator based on the long-tail distribution curve is executed by the processor, the following steps are also implemented:

[0207] The abrupt change point of the long-tailed distribution curve is determined by the slope of adjacent points in the long-tailed distribution curve.

[0208] The value of the static monitoring indicator corresponding to the mutation point is used as the critical value of the static monitoring indicator.

[0209] In one embodiment, when the logic of the computer program determining the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator based on the threshold and alarm value of the static monitoring indicator is executed by the processor, the following steps are also implemented:

[0210] Determine the difference between the critical value and the alarm value of the static monitoring indicator;

[0211] The ratio between the difference and the collection period of the dynamic monitoring indicator corresponding to the static monitoring indicator is used as the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator.

[0212] In one embodiment, when the computer program is executed by the processor, it also performs the following steps:

[0213] Based on the processing capacity of the business system, determine the first threshold of the static monitoring indicators of the business system.

[0214] Based on the system stability of the business system, determine the second threshold of the static monitoring indicators of the business system;

[0215] Based on the first threshold and the second threshold, determine the alarm values ​​of the static monitoring indicators of the business system.

[0216] In one embodiment, when the computer program is executed by the processor, it also performs the following steps:

[0217] If the actual value of a static monitoring indicator is found to be greater than the alarm value, and / or the actual value of a dynamic monitoring indicator is found to be greater than the monitoring threshold, then the business system is determined to be abnormal.

[0218] Obtain log data when the business system encounters an anomaly, and then feed the log data back to the operations and maintenance terminal.

[0219] In one embodiment, when the computer program is executed by the processor, it also performs the following steps:

[0220] If the actual value of a static monitoring indicator is found to be greater than the alarm value, the alarm level shall be determined according to the type of static monitoring indicator.

[0221] Alarms should be handled according to their alarm levels.

[0222] It should be noted that the data involved in this application (including but not limited to historical performance monitoring data, critical values ​​of static monitoring indicators, and alarm thresholds) are all information and data that have been authorized or fully authorized by all parties.

[0223] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0224] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0225] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for monitoring a business system, characterized in that, The method includes: Based on historical performance monitoring data of the business system, a long-tail distribution curve of the static monitoring indicators of the business system is constructed; based on the slope of adjacent points in the long-tail distribution curve, the abrupt change point of the long-tail distribution curve is determined; the value of the static monitoring indicator corresponding to the abrupt change point is taken as the critical value of the static monitoring indicator. Based on the critical values ​​and alarm values ​​of the static monitoring indicators, determine the monitoring thresholds of the dynamic monitoring indicators corresponding to the static monitoring indicators. The business system is monitored based on the alarm values ​​and monitoring thresholds; The step of determining the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator based on the critical value and alarm value of the static monitoring indicator includes: Determine the difference between the critical value and the alarm value of the static monitoring indicator; wherein, the alarm value of the static monitoring indicator is an alarm threshold of the static monitoring indicator that is preset according to the actual situation of the business system. The ratio between the difference and the collection period of the dynamic monitoring indicator corresponding to the static monitoring indicator is used as the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator.

2. The method according to claim 1, characterized in that, The method further includes: Based on the processing capacity of the business system, determine the first threshold of the static monitoring indicators of the business system. Based on the system stability of the business system, determine the second threshold of the static monitoring indicators of the business system; Based on the first threshold and the second threshold, the alarm values ​​of the static monitoring indicators of the business system are determined.

3. The method according to claim 1, characterized in that, The method further includes: If the actual value of the static monitoring indicator is found to be greater than the alarm value, and / or the actual value of the dynamic monitoring indicator is greater than the monitoring threshold, then the business system is determined to be abnormal. Obtain log data when the business system malfunctions, and feed the log data back to the operation and maintenance terminal.

4. The method according to claim 1, characterized in that, The method further includes: If the actual value of the static monitoring indicator is found to be greater than the alarm value, the alarm level is determined according to the type of the static monitoring indicator. According to the alarm level, perform alarm processing.

5. A business system monitoring device, characterized in that, The device includes: The critical value determination module is used to construct a long-tail distribution curve of the static monitoring indicators of the business system based on the historical performance monitoring data of the business system; determine the abrupt change point of the long-tail distribution curve based on the slope of adjacent points in the long-tail distribution curve; and take the value of the static monitoring indicator corresponding to the abrupt change point as the critical value of the static monitoring indicator. The threshold determination module is used to determine the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator based on the critical value and alarm value of the static monitoring indicator. The system monitoring module is used to monitor the business system based on the alarm values ​​and monitoring thresholds; The threshold determination module is specifically used to determine the difference between the critical value and the alarm value of the static monitoring indicator; wherein, the alarm value of the static monitoring indicator is an alarm threshold of the static monitoring indicator that is preset according to the actual situation of the business system; the ratio between the difference and the collection period of the dynamic monitoring indicator corresponding to the static monitoring indicator is used as the monitoring threshold of the dynamic monitoring indicator corresponding to the static monitoring indicator.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images