A method and system for accurately monitoring server response times

Abstract

This invention provides a method and system for accurately monitoring server response time, belonging to the field of distributed system performance monitoring. The invention first captures network traffic round-trip data packets related to the server-client session, parses the captured round-trip data packets to determine whether they are request or response traffic, and extracts their respective timestamps; it then determines whether the traffic originates from the server side or the client side based on the configuration file; if it originates from the server side, it calculates the actual server response time T based on the timestamps of the request and response traffic, assuming the time point when the client request information is captured is T1 and the time point when the server response information is captured is T2. response T2 - T1; if it comes from the client side, then calculate T. response The time interval is T2-T1-RTT, where RTT is the round-trip time for data packets. This invention improves the real-time performance, accuracy, and precision of server response time monitoring by determining the deployment location and performing differentiated calculations.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of distributed system performance monitoring, and particularly relates to a method and system for accurately monitoring server response time. BACKGROUND

[0002] With the deepening of digital transformation, the server as the core infrastructure of the enterprise directly determines the user experience and business continuity with its response time. Studies have shown that a web page loading delay of more than 2 seconds will increase the user churn rate by 47%, and the conversion rate of an e-commerce platform will decrease by 7% for every 100ms increase in application programming interface response time. Especially in the fields of finance, medicine and other fields with extremely high real-time requirements, server response delay may cause transaction failure or data synchronization interruption, resulting in direct economic losses. At the same time, with the popularity of Internet of Things, 5G communication, edge computing and other technologies, the server deployment environment is becoming increasingly complex, and cross-regional distribution and multi-node cooperation have become the norm. Traditional tools such as Ping and Traceroute can only measure network connectivity and are difficult to reflect application processing time; the analysis method based on application logs has problems of log delay, loss and intrusive modification. Therefore, accurate monitoring and analysis of server response time has become a core requirement for optimizing performance and ensuring service reliability.

[0003] In the prior art, the response time of the server is generally monitored by using traditional tools such as Ping and Traceroute. However, with the popularity of distributed systems, microservices and cloud native technologies, servers are mostly complex architectures with high concurrency, facing multi-level, cross-node request processing, and multiple links such as network, computing and storage may have a cumulative effect on the response time of the server, thereby causing fluctuations in the response time. The existing measurement method based on the above traditional monitoring tools does not consider the influence of different measurement positions (such as server side or client side) on the measurement results, resulting in data deviation. It can be seen that the existing server response time measurement has serious data deviation and is not accurate. SUMMARY

[0004] In view of the above defects or deficiencies in the prior art, the present application aims to provide a method and system for accurately monitoring server response time, which distinguishes whether the round-trip data packet comes from the server side or the client side, uses different server response time calculation methods for differential calculation, thereby accurately calculating the server response time, improving the accuracy, real-time performance and universality of the monitoring, and reducing the invasiveness of time monitoring.

[0005] To achieve the above purpose, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, the embodiments of the present application provide a method for accurately monitoring server response time, comprising the following steps:

[0007] Step S1, capturing network traffic round-trip packets related to the client-server session;

[0008] Step S2, respectively analyzing the captured round-trip packets, judging whether the packets are request traffic or response traffic according to the time sequence relationship, TCP flag bit and characteristic field of the packets, and extracting the respective time stamps;

[0009] Step S3, judging whether the traffic comes from the server side or the client side according to the pre-defined information in the configuration file; if it comes from the server side, entering Step S4; if it comes from the client side, entering Step S5;

[0010] Step S4, according to the extracted time stamps of the request traffic and the response traffic, setting the time point when the client request information is captured as T1 and the time point when the server response information is captured as T2, and calculating the actual response time T of the server as T2-T1; response

[0011] Step S5, according to the extracted time stamps of the request traffic and the response traffic, setting the time point when the client request information is captured as T1 and the time point when the server response information is captured as T2, and calculating the actual response time T of the server as T2-T1-RTT. response

[0012] As a preferred embodiment of the present application, Step S1 captures the network traffic round-trip packets related to the client-server session in a mirroring manner.

[0013] As a preferred embodiment of the present application, the mirroring manner is realized by setting a layer 2 manageable switch.

[0014] As a preferred embodiment of the present application, when Step S2 judges the packets according to the time sequence relationship, TCP flag bit and characteristic field of the packets, it specifically includes:

[0015] When the traffic target port is a standard port of the server or the first packet has a SYN field, the message header field contains a specified target server domain name, and the client device information is identified, it is determined that the packet is request traffic.

[0016] When the traffic target port is a temporary port or the TCP flag bit contains a SYN+ACK field, the message header field contains a server software field, and the format of the response body is declared, it is determined that the packet is response traffic.

[0017] As a preferred embodiment of the present application, the pre-defined content of the configuration file in Step S3 includes the type, protocol, port number, IP address of the application to which the traffic belongs, and the position of the traffic collection. ​​

[0018] In a second aspect, the embodiments of the present application further provide a system for accurately monitoring server response time, comprising a traffic collection module, a packet analysis module, a position identification module, a server scenario calculation module, a client scenario calculation module and a display module, wherein,

[0019] The traffic collection module is configured to capture network traffic round-trip packets associated with server-client session;

[0020] The packet analysis module is configured to analyze the captured round-trip packets respectively, determine whether the packets are request traffic or response traffic according to packet timing relationship, TCP flag and characteristic field, and extract respective time stamps;

[0021] The position identification module is configured to determine whether the traffic comes from the server side or the client side according to pre-defined information in the configuration file; if it comes from the server side, the server scenario calculation module is started; if it comes from the client side, the client scenario calculation module is started;

[0022] The server scenario calculation module is configured to calculate the actual server response time T response as T2-T1 according to the extracted time stamps of the round-trip packets, with the time point when the client request information is captured being T1 and the time point when the server response information is captured being T2, and send the calculated server response time T response to the display module;

[0023] The client scenario calculation module is configured to calculate the actual server response time T response as T2-T1-RTT according to the extracted time stamps of the round-trip packets, with the time point when the client request information is captured being T1, the time point when the server response information is captured being T2, and the round-trip delay of the round-trip packets being RTT, and send the calculated server response time T response to the display module;

[0024] The display module is configured to display the captured packets and the corresponding server response time.

[0025] As a preferred embodiment of the present application, the traffic collection module captures the network traffic round-trip packets associated with server-client session in a mirroring manner.

[0026] As a preferred embodiment of the present application, the traffic collection module is implemented through a layer 2 manageable switch.

[0027] As a preferred embodiment of the present application, the packet analysis module is further configured to:

[0028] The flow target port is a standard port of the server or the first data packet has a SYN field, the message header field contains a specified target server domain name, and the client device information is identified, and it is determined that the data packet is request flow.

[0029] The flow target port is a temporary port or the TCP flag bit contains a SYN+ACK field, the message header field contains a server software field, and the format of the response body is declared, and it is determined that the data packet is response flow.

[0030] As a preferred embodiment of the present application, the configuration file in the position recognition module includes the type of the application to which the flow belongs, the protocol, the port number, the IP address, the URL information, and the position of the collected flow.

[0031] The technical solution provided by the embodiment has the following beneficial effects:

[0032] The method and system for accurately monitoring the server response time provided by the embodiment first analyze the request flow and the response flow in the captured network flow round-trip data packet related to the server and client session, and extract the respective time stamps; then, according to the configuration file matching judgment, it is confirmed whether the data packet comes from the client side or the server side, and the appropriate algorithm is matched to accurately calculate the server response time; when it comes from the server side, the difference between the time point of the server response captured and the time point of the client request information captured is used as the actual server response time; when it comes from the client side, the difference between the time point of the server response information captured and the time point of the client request information captured is used, and then the round-trip delay of the round-trip data packet is subtracted, as the calculation of the actual server response time. The present application improves the accuracy of the monitoring result by judging the deployment position and performing differential calculation, breaks through the limitation of traditional tools (such as Ping) that only measure the network layer delay, realizes the end-to-end full link time consumption decomposition, improves the accuracy, real-time and universality of the server response time monitoring, and reduces the invasiveness of the time monitoring.

[0033] Of course, implementing any product or method of the present application does not necessarily require all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0035] Figure 1 The figure is a flow chart of the method for accurately monitoring the server response time according to the embodiment of the present application. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present application will be apparently and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0037] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. In the description of the present application, the terms "first", "second", "third", "fourth" and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0038] Based on the monitoring problem of the server response time, the embodiments of the present application provide a method and system for accurately monitoring the server response time. After the traffic access, the traffic analysis is performed through the configuration file to obtain the monitoring position, and the server side and the client side are distinguished. Differentiated calculation is performed based on different positions, and the server response time is calculated respectively. At the same time, the different characteristics of each position are fully considered during the calculation, so as to realize the accurate monitoring of the server response time.

[0039] As shown in Figure 1 The method for accurately monitoring the server response time comprises the following steps:

[0040] Step S1, capturing network traffic round-trip data packets related to the server and client session.

[0041] In this step, for the related network traffic round-trip data packets, the mirroring method is used for capturing. When implemented, the mirroring function of the network traffic round-trip data packets can be realized by setting a layer 2 manageable switch. Through the mirroring operation, different source traffic information is copied to the specified port.

[0042] When implemented, the traffic collection module is set to collect traffic, that is, to capture data packets. When capturing data packets, there are two cases. When the monitored object is on the client side, the traffic collection module is deployed on the client side. When the monitored object is on the server side, the traffic collection module is deployed on the server side. The specific deployment of the traffic collection module is deployed according to the actual monitoring situation.

[0043] Step S2, respectively analyze the captured round-trip data packets, according to the data packet timing relationship, TCP flag and characteristic field to determine whether the data packet is request traffic or response traffic, and extract the respective time stamp.

[0044] In this step, when judging the data packet according to the data packet timing relationship, TCP flag and characteristic field, specifically includes:

[0045] The traffic target port is the standard port of the server or the first data packet has the SYN field, and the message header field contains the specified target server domain name, and the identification of the client device information, and the current data packet is determined as request traffic;

[0046] The traffic target port is a temporary port or the TCP flag contains the SYN+ACK field, and the message header field contains the identification of the server software field, and the format of the response body is declared, and the data packet is determined as response traffic.

[0047] Steps S1 and S2 capture the round-trip data packets of network traffic and analyze and extract the time stamp, realize the real-time monitoring of the server response time, and break through the limitation of traditional tools (such as Ping) only measuring network layer delay. At the same time, in the analysis of the data packet, the data packet timing relationship, TCP flag and characteristic field and other standards are introduced to determine whether the data packet is request traffic or response traffic, and the accuracy of the determination is improved.

[0048] Step S3, according to the pre-defined information in the configuration file, determine whether the traffic comes from the server side or the client side; if it comes from the server side, go to step S4; if it comes from the client side, go to step S5.

[0049] In this step, according to the configuration file, automatically determine whether the traffic comes from the server side or the client side; in the configuration file, predefinition can be made, and the predefinition content includes the type of the application to which the traffic belongs, the protocol, the port number, the IP address, the URL information and the position of the collected traffic. Among them, the position of the collected traffic is the deployment position of the traffic collection module, which is generally defaulted to be on the server side; if the traffic collection module is located on the client side, it can be defined as the client side. The purpose of the configuration file is to dynamically adjust the calculation method of the server response time according to the deployment position of the traffic collection module. Different calculation formulas are referred to for different deployment positions, so as to more accurately calculate the server response time. In this step, the configuration file is used to predefine the type of the application to which the traffic belongs, the protocol, the port number, the IP address and other information, which can dynamically adjust the monitoring strategy according to different deployment positions, so as to improve the flexibility and adaptability of the monitoring.

[0050] Step S4, according to the extracted request traffic and response traffic timestamp, set the time point of the client request information captured as T1, the time point of the server response information captured as T2, calculate the server actual response time T response T2-T1.

[0051] In this step, the server actual response time calculation formula is:

[0052] Server actual response time = the time point of the client request information captured T1 and the time point of the server response captured T2; that is

[0053] T response =T2-T1

[0054] In the server side scenario: the client initiates a TCP payload non-0 server request, and the server responds to the request at the application layer, at this time, it is considered that both are in the same LAN, so the network delay is ignored.

[0055] Step S5, according to the extracted request traffic and response traffic timestamp, set the time point of the client request information captured as T1, the time point of the server response information captured as T2, the round-trip delay of the round-trip packet as RTT; calculate the server actual response time T response T2-T1-RTT.

[0056] In this step, the server actual response time T response The calculation formula is as follows:

[0057] T response =T2-T1-RTT

[0058] In the client side scenario, it belongs to a public network environment with large fluctuations, at this time, the server response time includes the network delay, and the network round-trip delay (Round-Trip Time, RTT) is deducted to eliminate the influence of the transmission path on the response time, so as to realize accurate monitoring of the server response time.

[0059] As can be seen, the embodiment of the application calculates the server response time according to the difference of traffic sources (server side or client side), directly calculates the response time on the server side, and considers the round-trip delay (RTT) on the client side, so as to improve the accuracy of monitoring; at the same time, the data monitoring is carried out by a non-invasive method, avoiding the interference to the normal business process, and improving the reliability of monitoring.

[0060] Based on the same idea, the embodiment of the application also provides a system for accurately monitoring the server response time, which comprises: a traffic collection module, a packet analysis module, a position identification module, a server scenario calculation module, a client scenario calculation module and a display module.

[0061] The traffic collection module is configured to capture network traffic round-trip data packets associated with the server-client session;

[0062] The data packet analysis module is configured to analyze the captured round-trip data packets respectively, to determine whether the data packets are request traffic or response traffic according to the time sequence relationship, TCP flag bits and characteristic fields of the data packets, and to extract respective time stamps;

[0063] The position identification module is configured to determine whether the traffic comes from the server side or the client side according to the pre-defined information in the configuration file, to start the server scenario calculation module if the traffic comes from the server side, and to start the client scenario calculation module if the traffic comes from the client side;

[0064] The server scenario calculation module is configured to calculate the actual server response time T response as T2-T1 and to send the calculated server response time T response to the display module;

[0065] The client scenario calculation module is configured to calculate the actual server response time T response as T2-T1-RTT and to send the calculated server response time T response to the display module;

[0066] The display module is configured to display the captured data packets and the corresponding server response time.

[0067] The modules in the embodiments are implemented by a processor, and a memory is appropriately increased when storage is needed. The processor can be, but is not limited to, a microprocessor (MPU), a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the like. The memory can include a random access memory (RAM) and can also include a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0068] In the above embodiments, the implementation can be entirely or partially achieved by software, hardware, firmware, or any combination thereof. When implemented by software, the implementation can be entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the entire or partial flow or function described in the embodiments of the present application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode.

[0069] In addition, it should be noted that the server response time monitoring system and the server response time monitoring method described in the embodiments are corresponding. The description and limitation of the method are also applicable to the system, and will not be repeated here.

[0070] It can be seen from the above technical solutions that the method and system for accurately monitoring server response time provided by the embodiments of the present application first analyze the request flow and response flow in the round-trip data packet and extract the respective time stamps; then, according to the configuration file matching judgment, it is confirmed whether the data packet comes from the client side or the server side, and the appropriate algorithm is matched to accurately calculate the response time of the server; when it comes from the server side, the difference between the time point of the captured server response and the time point of the captured client request information is used as the actual response time of the server; when it comes from the client side, the difference between the time point of the captured server response information and the time point of the captured client request information is used, and then the round-trip delay of the round-trip data packet is subtracted, as the calculation of the actual response time of the server. The present application improves the accuracy of the monitoring result by judging the deployment position and performing differential calculation, breaks through the limitation of traditional tools (such as Ping) that only measure network layer delay, realizes end-to-end full link time consumption decomposition, improves the accuracy, real-time and universality of server response time monitoring, and at the same time reduces the invasiveness of time monitoring.

[0071] The above description is only the preferred embodiments of the present application and the explanation of the applied technical principles, and is not intended to limit the scope of the claimed application, but only represents the preferred embodiments of the present application. Those skilled in the art should understand that the scope of the application involved in the present application is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

Claims

1. A method for accurately monitoring server response time, characterized in that, Includes the following steps: Step S1: Capture network traffic round-trip packets related to the server-client session; Step S2: parse the captured round-trip data packets respectively, determine whether the data packets are request traffic or response traffic based on the data packet timing relationship, TCP flag bits and feature fields, and extract their respective timestamps; Step S3: Determine whether the traffic comes from the server side or the client side based on the predefined information in the configuration file; If the error originates from the server, proceed to step S4; if it originates from the client, proceed to step S5. Step S4: Based on the timestamps of the extracted request and response traffic, let T1 be the time point when the client request information is captured and T2 be the time point when the server response information is captured, and calculate the actual server response time T. response For T2-T1; Step S5: Based on the timestamps of the extracted request traffic and response traffic, let the time point when the client request information is captured be T1, the time point when the server response information is captured be T2, and the round-trip time of the round-trip data packets be RTT. Calculate the actual server response time T response For T2-T1-RTT.

2. The method for accurately monitoring server response time according to claim 1, characterized in that, Step S1 captures network traffic round-trip packets related to the server and client sessions using a mirroring method.

3. The method for accurately monitoring server response time according to claim 2, characterized in that, Mirroring is achieved by configuring a Layer 2 manageable switch.

4. The method for accurately monitoring server response time according to claim 1, characterized in that, Step S2, which involves judging data packets based on packet timing, TCP flags, and characteristic fields, specifically includes: If the destination port of the traffic is the standard port of the server, or if the first data packet contains a SYN field, a header field containing the domain name of the specified target server, and information identifying the client device, then the data packet is determined to be request traffic. When the destination port of the traffic is an ephemeral port, or the TCP flags contain the SYN+ACK field, the header fields contain fields identifying the server software, and the format of the response body is declared, the data packet is determined to be response traffic.

5. The method for accurately monitoring server response time according to claim 4, characterized in that, The predefined information in the configuration file mentioned in step S3 includes the type of application to which the traffic belongs, the protocol, port number, IP address, URL information, and the location where the traffic is collected.

6. A system for accurately monitoring server response time, characterized in that, The system includes: a traffic acquisition module, a data packet parsing module, a location identification module, a server scene calculation module, a client scene calculation module, and a display module; wherein, The traffic acquisition module is used to capture round-trip network traffic data packets related to server and client sessions. The packet parsing module is used to parse the captured round-trip packets respectively, determine whether the packets are request traffic or response traffic based on the packet timing relationship, TCP flags and feature fields, and extract their respective timestamps; The location identification module is used to determine whether the traffic comes from the server side or the client side based on the predefined information in the configuration file; if it comes from the server side, the server scene calculation module is started; if it comes from the client side, the client scene calculation module is started. The server scenario calculation module is used to calculate the actual server response time T based on the timestamps of the extracted request and response traffic, assuming the time point when the client request information is captured is T1 and the time point when the server response information is captured is T2. response Let T2-T1 be the calculated server response time T. response Send to the display module; The client-side scenario calculation module is used to calculate the actual server response time T based on the timestamps of the extracted request and response traffic, assuming the time point when the client request information is captured is T1, the time point when the server response information is captured is T2, and the round-trip time of the round-trip data packet is RTT. response Let T2 - T1 - RTT and calculate the server response time T. response Send to the display module; The display module is used to display the captured data packets and the corresponding server response time.

7. The system for accurately monitoring server response time according to claim 6, characterized in that, The traffic acquisition module uses a mirroring method to capture network traffic round-trip data packets related to server and client sessions.

8. The system for accurately monitoring server response time according to claim 7, characterized in that, The traffic acquisition module is implemented through a Layer 2 manageable switch.

9. The system for accurately monitoring server response time according to claim 6, characterized in that, The packet parsing module is further used for: If the destination port of the traffic is the standard port of the server, or if the first data packet contains a SYN field, a header field containing the domain name of the specified target server, and information identifying the client device, then the data packet is determined to be request traffic. When the destination port of the traffic is an ephemeral port, or the TCP flags contain the SYN+ACK field, the header fields contain fields identifying the server software, and the format of the response body is declared, the data packet is determined to be response traffic.

10. The system for accurately monitoring server response time according to claim 9, characterized in that, The predefined information in the configuration file of the location identification module includes the type of application to which the traffic belongs, the protocol, port number, IP address, URL information, and the location where the traffic is collected.

Images