A log tracing method and apparatus based on load balancing equipment

Abstract

This invention discloses a log tracing method and apparatus based on a load balancer. The method includes: determining the log tracing range based on a user-issued log capture command; real-time monitoring of network service ports using the load balancer to create a session; obtaining the corresponding source IP address and port number based on the log tracing range and the client connection request; constructing a log recording module object; and associating the client connection object, its corresponding derived connection object, and the server-side connection object with the log recording module object through a pointer to the log recording module object; responding to and processing the client's operation request based on the load balancer and the session; and recording or outputting the log data of the current log recording point at the current log level. By applying the technical solution provided by this invention, the amount of log data recorded and the system performance pressure are reduced.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and more specifically, to a log tracing method and apparatus based on a load balancing device. Background Technology

[0002] In today's booming internet technology landscape, load balancing systems occupy a crucial position in the field of computer network equipment, and their performance stability is vital for the normal operation of backend services. Under high load scenarios, performance fluctuations in load balancing devices can negatively impact the stability of backend services. For example, during e-commerce promotional events, a large number of users simultaneously access the e-commerce platform. Load balancing devices need to handle massive amounts of requests; if their performance is unstable, some users may be unable to access product pages, experience order failures, and other problems, severely impacting user experience and merchant business. This makes the performance stability of load balancing systems highly sensitive; under high load scenarios, performance fluctuations in load balancing devices can affect the stability of backend services.

[0003] Meanwhile, the business logic of the user-space load balancing system is highly complex. When the system malfunctions, packet capture analysis alone is insufficient to accurately pinpoint the specific cause. While packet capture analysis can obtain partial information about network packets, it cannot fully reveal the details of the load balancing system's internal processing flow, such as how data is forwarded between different links and the processing status of each stage. Therefore, it is essential to build a logging module to record the detailed processing flow of the load balancing system.

[0004] However, traditional logging modules record all processing steps of the load balancing system, resulting in a massive log volume. This excessive log writing incurs significant performance overhead; the continuous generation of log content in memory and its frequent writing to disk slows down the load balancing system's processing capabilities, leading to severe performance fluctuations. Consequently, under high load, the logging system often becomes ineffective. The main problem lies in the lack of any restrictions on the content recorded in the logs. Therefore, traditional logging modules have a significant flaw: they indiscriminately record all processes in the load balancing system's processing flow, resulting in a huge log volume. The numerous log write operations generate high performance overhead; the continuous generation of log content in memory and frequent writing to disk severely slows down the load balancing system's processing capabilities. Under high load, this performance fluctuation caused by the overhead is even more pronounced, rendering the logging system ineffective at critical times and unable to provide effective support for troubleshooting. The root cause of this problem is that traditional logging modules do not reasonably limit the content recorded in the logs, failing to consider the high workload of the load balancing system and the resulting performance pressure.

[0005] Therefore, it is necessary to introduce a new method and apparatus that can selectively record and store log data of the load balancing system's processing flow without affecting the system's performance. This allows for rapid troubleshooting and location of log data related to the relevant links when a load balancing system malfunctions. This addresses the technical problems of existing technologies, such as massive log volume, slowing down the load balancing system's processing capacity, severe performance fluctuations, and inability to provide effective support for troubleshooting. The goal is to reduce the amount of log data and system performance pressure, improve the accuracy of log recording and the efficiency of log-based troubleshooting, thereby enhancing the overall performance and stability of the load balancing system. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a log tracing method and apparatus based on load balancing devices. By defining the log recording module object structure and log tracing scope, and obtaining the corresponding source IP address and port number based on the log tracing scope and client connection requests, a log recording module object is constructed for each client connection request. Based on the constructed log recording module object and the log level of the current log recording point, log data of the load balancing system's processing flow is recorded. This achieves selective recording and storage of log data of the load balancing system's processing flow without affecting the load balancing system's performance. Furthermore, it allows for rapid troubleshooting and location of log data related to the relevant links when a load balancing system failure occurs. This solves the technical problems of existing technologies, such as massive log recording volumes, slowing down the load balancing system's processing capabilities, and severe performance fluctuations that fail to provide effective support for fault diagnosis. Therefore, it reduces the amount of log data and system performance pressure, improves the accuracy of log recording and the efficiency of log-based fault diagnosis, and enhances the overall performance and stability of the load balancing system.

[0007] This invention provides a log tracing method based on a load balancing device. The method includes: S1, determining the log tracing scope: defining a log recording module object structure, receiving and parsing user-issued log tracing capture instructions in real time, and determining the log tracing scope of client connection objects to be traced based on the parsing result of the log tracing capture instructions; S2, client monitoring: based on the load balancing device, real-time monitoring of network service ports, obtaining the source IP address, port number, and packet corresponding to the monitored client connection requests, creating a client connection object based on the packet corresponding to the client connection request, and determining the server-side connection object based on the load balancing strategy provided by the load balancing device and the client connection object, and creating a session; S3, the log recording module... Image generation and association: Based on the log tracking range and the corresponding source IP address and port number obtained from the client connection request, a log recording module object is constructed. The client connection object, its corresponding derived connection object, and the server-side connection object are associated with the log recording module object via the log module object pointer, and the value of the log module object pointer is updated. S4, Log recording and storage: Based on the load balancing device and the session response, the client's operation request is processed. The log data of the current log recording point is recorded or output in real time according to the log level of the current log recording point processed by the operation request, as well as the log recording level and log dump level of the log recording module object. The log tracking range includes the IP address subnet mask and port number.

[0008] Preferably, in step S1, the step of determining the log tracking range further includes: S11, defining the log recording module object structure: defining the log recording module object structure according to the content of the log records, wherein the log recording module object structure includes: log recording level, log dump level, and log storage buffer; S12, determining the log tracking range of client connections: based on the load balancing device, receiving the log tracing instructions issued by the user in real time, parsing the log tracing instructions, obtaining the log recording level, log dump level, storage capacity of the log storage buffer, and the IP address subnet mask and port number of the client connection object that needs to record logs in the log tracing instructions, and determining the client subnet address and port number of the client connection that needs to be log tracked based on the obtained IP address subnet mask and port.

[0009] Preferably, in step S2, the client listening step includes: S21, based on the load balancing device, real-time monitoring of the network service port; when a client connection request is detected, receiving the client connection request, parsing the client connection request, obtaining the source IP address and source port number in the SYN packet corresponding to the client connection request, and determining the client subnet address and port number corresponding to the client connection request based on the source IP address and source port number; S22, determining and creating a client connection object based on the SYN packet corresponding to the client connection request, and initializing the log module object pointer of the client connection object; determining a server-side connection object based on the destination IP address and destination port number corresponding to the client connection request, and initializing the log module object pointer of the server-side connection object; S23, based on the client connection object and server-side connection object corresponding to the client connection request, determining whether a connection object already exists that corresponds to the client connection request. If a corresponding session exists for the client connection request, the client connection request is forwarded based on the corresponding session and load balancing strategy, and a log is recorded. If no corresponding session exists, a new session is created. The SYN packet includes the source IP address, source port number, destination IP address, destination port number, and hash function. The client connection object is used to acquire and store the TCP 5-tuple data sent by the client, which includes the source IP address, source port number, destination IP address, destination port number, and protocol type. The initial values ​​of the log module object pointers of both the client connection object and the server connection object are null. The session records the correspondence between the client connection object and the server connection object, allowing for quick task forwarding between them during load balancing.

[0010] Preferably, in step S3, the step of constructing a log recording module object based on the log tracking range and the corresponding source IP address and port number obtained from the client connection request further includes: comparing the client subnet address and port number corresponding to the client connection request with the IP address subnet mask and port number of the client connection object that needs to be logged, respectively. S311, if the IP address subnet mask of the client connection object that needs to be logged contains a client subnet address that is the same as the client subnet address corresponding to the client connection request, and the port number of the client connection object that needs to be logged contains a port number that is the same as the port number corresponding to the client connection request, then according to the log recording module object structure, a log recording module object is created for the client connection request, and the log recording level and log are determined. At the dump level, the log storage buffer corresponding to the client connection request is initialized according to the storage capacity of the log storage buffer; S312, if the subnet mask of the IP address of the client connection object that needs to record logs does not contain a client subnet address that is the same as the client subnet address corresponding to the client connection request, or if the port number of the client connection object that needs to record logs does not contain a port number that is the same as the port number corresponding to the client connection request, then the client connection request does not need to be logged, and the client connection request is forwarded to the server based on the load balancing device, load balancing policy and session, as well as the destination IP address and destination port number corresponding to the client connection request; wherein, the data types of the log recording level and log dump level of the log recording module object corresponding to the client connection request are both integers.

[0011] Preferably, in step S3, the step of associating the client connection object, the corresponding derived connection object, and the server-side connection object with the log recording module object through the log module object pointer further includes: when the IP address subnet mask of the client connection object that needs to record logs contains a client subnet address that is the same as the client subnet address corresponding to the client connection request, and the port number of the client connection object that needs to record logs contains a port number that is the same as the port number corresponding to the client connection request, the log module object pointer of the client connection object corresponding to the client connection request is set to point to the log recording module object, and the date of the client connection object is updated. The value of the log module object pointer is set; the log module object pointer of the server-side connection object corresponding to the client connection request is set to the log recording module object, and the value of the log module object pointer of the server-side connection object is updated; if the client connection object has a derived connection object, the log module object pointer of the derived connection object of the client connection object is set to the log recording module object, and the value of the log module object pointer of the derived connection object of the client connection object is updated; wherein, the derived connection object corresponding to the client connection object is the connection object corresponding to the server-side connection object determined according to the load balancing strategy, or the connection object corresponding to the server-side connection object obtained by executing the script related to the load balancing strategy.

[0012] Preferably, in step S4, the step of recording or outputting the log data of the current log point in real time according to the log level of the current log point processed by the operation request, and the log level and log dump level of the log recording module object, specifically includes: obtaining the log level of the current log point, comparing the log level of the current log point with the log level of the log recording module object, and if the log level of the current log point is greater than or equal to the log level of the log recording module object, then recording the processing operation log data of the client operation request of the previous log point into the log storage buffer of the log recording module object; if the log level of the current log point is less than .... The logging level of the logging module object does not record the processing operation logs of client operation requests at previous logging points. A logging point is a recording point set up to record or output the processing operation logs of client operation requests. Each logging point has a pre-set corresponding logging level. The logging points are set according to the client connection object, server connection object, and the derived connection objects corresponding to the client connection object. Through the logging point, the client connection object, server connection object, or derived connection object corresponding to the logging point can be obtained. The log storage buffer of the logging module object is a fixed-length memory space used to temporarily store the processing operation logs of client operation requests.

[0013] Preferably, in step S4, the step of recording or outputting the log data of the current log point in real time according to the log level of the current log point processed by the operation request, as well as the log level and log dump level of the log recording module object, further includes a log dumping step, specifically: comparing the log level of the current log point with the log dump level of the log recording module object; if the log level of the current log point is greater than or equal to the log dump level of the log recording module object, then storing the client operation request processing operation log data stored in the log storage buffer of the log recording module object to the disk, and clearing the log storage buffer of the log recording module object; if the log level of the current log point is less than or equal to the log dump level of the log recording module object, then performing forced log caching and dumping processing according to the amount of client operation request processing operation log data stored in the log storage buffer of the log recording module object.

[0014] Preferably, in step S4, the step of performing forced caching and dumping of logs based on the amount of client operation request processing operation log data stored in the log storage buffer of the log recording module object specifically involves: if the amount of client operation request processing operation log data stored in the log storage buffer of the log recording module object is equal to the capacity of the log storage buffer of the log recording module object, then the client operation request processing operation log data stored in the log storage buffer of the log recording module object is stored in the disk, and the log storage buffer of the log recording module object is cleared.

[0015] Accordingly, the present invention also provides a log tracing device based on a load balancer, the device comprising a log tracing range processing module, a client listening module, a log recording module object association module, and a log recording storage module; wherein,

[0016] The log tracking range processing module is used to determine the log tracking range, define the log recording module object structure, receive and parse the tracking log capture command issued by the user in real time, and determine the log tracking range of the client connection object that needs to be tracked based on the parsing result of the tracking log capture command.

[0017] The client listening module is used for client listening. Based on the load balancing device, it listens to the network service port in real time, obtains the source IP address, port number and packet corresponding to the client connection request according to the listened client connection request, creates a client connection object according to the packet corresponding to the client connection request, determines the server-side connection object according to the load balancing policy provided by the load balancing device and the client connection object, and creates a session.

[0018] The log recording module object association module is used to generate and associate log recording module objects. Based on the log tracking range and the corresponding source IP address and port number obtained from the client connection request, the log recording module object is constructed. The client connection object, as well as the derived connection object and server connection object corresponding to the client connection object, are associated with the log recording module object through the log module object pointer, and the value of the log module object pointer is updated.

[0019] The log recording and storage module is used for log recording and storage. Based on the load balancing device and the session response, it processes the client's operation requests and records or outputs the log data of the current log recording point in real time according to the log level of the current log recording point processed by the operation request, as well as the log recording level and log dump level of the log recording module object.

[0020] The log tracking scope includes the IP address subnet mask and port number.

[0021] This invention, by applying the above technical solutions, defines the structure of a log recording module object and the scope of log tracking. Based on the log tracking scope and the corresponding source IP address and port number obtained from the client connection request, a log recording module object is constructed for the client connection request. Based on the constructed log recording module object and the log level of the current log recording point, log data of the load balancing system's processing flow is recorded. This achieves selective recording and storage of log data of the load balancing system's processing flow without affecting the load balancing system's performance. Furthermore, it enables rapid troubleshooting and location of log data related to the relevant links when the load balancing system malfunctions. This solves the technical problems of existing technologies, such as massive log recording volumes, slowing down the load balancing system's processing capabilities, and severe performance fluctuations that fail to provide effective support for fault diagnosis. Therefore, it reduces the amount of log data and system performance pressure, improves the accuracy of log recording and the efficiency of log-based fault diagnosis, and enhances the overall performance and stability of the load balancing system. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating a log tracing method based on a load balancing device proposed in an embodiment of the present invention is shown.

[0024] Figure 2 This diagram illustrates the response and processing flow of a log tracing method based on a load balancing device proposed in an embodiment of the present invention.

[0025] Figure 3 A schematic diagram of a log tracking device based on a load balancing device, as proposed in an embodiment of the present invention, is shown. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] This invention provides a log tracing method based on a load balancer, such as... Figure 1 and Figure 2 As shown, the method includes the following steps:

[0028] S1, Determine the log tracking scope: Define the log recording module object structure, receive and parse the user's log tracking capture command in real time, and determine the log tracking scope of the client connection object that needs to be tracked based on the parsing result of the log tracking capture command;

[0029] in,

[0030] The log tracking scope includes the IP address subnet mask and port number.

[0031] In this embodiment, step S1, the step of determining the log tracking range, further includes:

[0032] S11, Define the log recording module object structure: Define the log recording module object structure according to the content of the log recording, wherein the log recording module object structure includes: log recording level, log dump level and log storage buffer;

[0033] S12, Determine the log tracking scope of the client connection: Based on the load balancing device, receive the tracking log capture command issued by the user in real time, parse the tracking log capture command, obtain the log recording level, log dump level, log storage buffer storage capacity, and IP address subnet mask and port number of the client connection object that needs to be logged in the tracking log capture command, and determine the client subnet address and port number of the client connection that needs to be logged based on the obtained IP address subnet mask and port.

[0034] It should be noted that this step, by determining the scope of log tracking for client connections, enables the identification of which client connections can be logged based on filtering criteria. The filtering criteria are subnet address and port number. The reason for filtering here is to address the performance jitter issue mentioned in the background technology. Because the load balancer handles a large volume of traffic, logging all operations would significantly reduce the load balancer's capacity. By filtering by subnet address and port number, the scope of log recording is greatly reduced.

[0035] The logging module object is a structure whose main contents are an integer log level and an integer dump level. It associates the corresponding connection object with the logging module object via pointers. The connection object is a structure representing a TCP connection, such as... Figure 2As shown, the client connection object and the server connection object are key components of a load balancer. One of the main tasks of the load balancer is to forward data from client connections to the server. This forwarding operation uses connection objects to obtain and send relevant data. When a client sends data, the connection object is identified by the TCP 5-tuple data, which includes the source IP address, source port number, destination IP address, destination port number, and protocol type. At this point, the client connection object is obtained. A log module object pointer is then used on the client connection object to point to the newly generated log module object. Thus, the system visualizes the filtering parameters as a log structure. Only connection objects with this structure pointer record logs at the specific log level described in the log object.

[0036] Associating the derived connection object of the client connection object with the log module object is crucial. As mentioned above, the main task of the load balancer is to forward client data to the server according to a certain load balancing strategy through the connection object. Upon receiving a client connection creation request, it executes certain load balancing strategies. After executing the strategies, it will inevitably obtain a server connection object corresponding to the client connection object described in the second step, thus achieving forwarding. However, when logging, since the behavior of both the server connection object and the client connection object jointly determines the process of this forwarding task, it is necessary to record the behavior of the server connection object corresponding to the client connection object in the log to ensure the completeness of the log content. Therefore, after determining the server connection object, a pointer to the client's log module object is also needed, which provides the foundation for the next step.

[0037] Meanwhile, during the forwarding process of the load balancer, there are some key points that require logging to output critical information. This information can be used to infer and locate the cause of the fault in the event of a subsequent failure. These points are called log recording points. Each log recording point has a log level. Logs are only printed and recorded when the log level of the log recording point is greater than or equal to the log level of the log module object. Logs are only stored on disk when the log level of the log recording point is greater than or equal to the log dump level of the log recording module object. This reduces the number of disk operations and improves storage performance. Connection objects can usually be obtained at log recording points. The log objects on the connection objects (which can be client or server connection objects, depending on the location of the log recording point) are used to determine whether logs need to be recorded and dumped. Because only client connection objects and related server connection objects that meet the conditions have log objects (not null pointers), only these related log information will be output.

[0038] S2, Client Listening: Based on the load balancing device, the network service port is monitored in real time. The source IP address, port number and packet corresponding to the client connection request are obtained according to the monitored client connection request. A client connection object is created according to the packet corresponding to the client connection request. The server-side connection object is determined according to the load balancing policy provided by the load balancing device and the client connection object, and a session is created.

[0039] In this embodiment, step S2, the client listening step includes:

[0040] S21, Based on the load balancing device, the network service port is monitored in real time. When a client connection request is detected, the client connection request is received and parsed to obtain the source IP address and source port number in the SYN packet corresponding to the client connection request. The client subnet address and port number corresponding to the client connection request are determined based on the source IP address and source port number.

[0041] S22, determine and create a client connection object based on the SYN packet corresponding to the client connection request, and initialize the log module object pointer of the client connection object;

[0042] The server-side connection object is determined based on the destination IP address and destination port number corresponding to the client connection request, and the log module object pointer of the server-side connection object is initialized.

[0043] S23. Based on the client connection object and server connection object corresponding to the client connection request, determine whether a session corresponding to the client connection request already exists. If a session corresponding to the client connection request exists, forward the client connection request based on the session corresponding to the client connection request and the load balancing strategy, and record the log. If no session corresponding to the client connection request exists, create a new session.

[0044] in,

[0045] The SYN packet includes the source IP address, source port number, destination IP address, destination port number, and hash function;

[0046] The client connection object is used to acquire and store TCP 5-tuple data sent by the client. The TCP 5-tuple data includes source IP address, source port number, destination IP address, destination port number, and protocol type.

[0047] The initial values ​​of the log module object pointer of the client connection object and the log module object pointer of the server connection object are both null.

[0048] The session is used to record the correspondence between client connection objects and server connection objects. When performing load balancing task forwarding, the task forwarding between client connection objects and server connection objects can be quickly completed through the session.

[0049] S3, Log recording module object generation and association: Based on the log tracking range and the corresponding source IP address and port number obtained from the client connection request, a log recording module object is constructed. The client connection object, as well as the derived connection object and server-side connection object corresponding to the client connection object, are associated with the log recording module object through the log module object pointer, and the value of the log module object pointer is updated.

[0050] In this embodiment, step S3, the step of constructing the log recording module object based on the log tracking range and the corresponding source IP address and port number obtained from the client connection request, further includes:

[0051] The client subnet address and port number corresponding to the client connection request are compared with the IP address, subnet mask, and port number of the client connection object that needs to be logged.

[0052] S311 If the IP address subnet mask of the client connection object that needs to record logs contains a client subnet address that is the same as the client subnet address corresponding to the client connection request, and the port number of the client connection object that needs to record logs contains a port number that is the same as the port number corresponding to the client connection request, then according to the log recording module object structure, a log recording module object is created for the client connection request, and the log recording level and log dump level are determined. The log storage buffer corresponding to the client connection request is initialized according to the storage capacity of the log storage buffer.

[0053] S312, if the IP address subnet mask of the client connection object that needs to be logged does not contain a client subnet address that is the same as the client subnet address corresponding to the client connection request, or if the port number of the client connection object that needs to be logged does not contain a port number that is the same as the port number corresponding to the client connection request, then the client connection request does not need to be logged, and the client connection request is forwarded to the server based on the load balancing device, load balancing policy and session, as well as the destination IP address and destination port number corresponding to the client connection request;

[0054] In this context, the data types of the log recording level and log dump level of the log recording module object corresponding to the client connection request are both integers.

[0055] In this embodiment, step S3, which involves associating the client connection object, the corresponding derived connection object, and the server connection object with the log recording module object via the log module object pointer, further includes:

[0056] When the IP address subnet mask of the client connection object that needs to be logged contains a client subnet address that is the same as the client subnet address corresponding to the client connection request, and the port number of the client connection object that needs to be logged contains a port number that is the same as the port number corresponding to the client connection request, the log module object pointer of the client connection object corresponding to the client connection request is set to point to the log recording module object, and the value of the log module object pointer of the client connection object is updated.

[0057] Set the log module object pointer of the server-side connection object corresponding to the client connection request to the log recording module object, and update the value of the log module object pointer of the server-side connection object;

[0058] By updating the value of the log module object pointer of the server-side connection object, the client connection object and the server-side connection object corresponding to the client connection request are respectively associated with the log recording module object.

[0059] If the client connection object has a derived connection object, the log module object pointer of the derived connection object of the client connection object is made to point to the log recording module object, and the value of the log module object pointer of the derived connection object of the client connection object is updated.

[0060] By updating the value of the log module object pointer of the derived connection object of the client connection object, the association between the derived connection object corresponding to the client connection object and the log recording module object is realized.

[0061] The derived connection object corresponding to the client connection object is either the connection object corresponding to the server connection object determined according to the load balancing strategy, or the connection object corresponding to the server connection object obtained by executing a script related to the load balancing strategy.

[0062] S4, Log recording and storage: Based on the load balancing device and the session response, process the client's operation request, and record or output the log data of the current log recording point in real time according to the log level of the current log recording point processed by the operation request, as well as the log recording level and log dump level of the log recording module object.

[0063] In this embodiment, step S4, which involves recording or outputting the log data of the current log recording point in real time based on the log level of the current log recording point processed by the operation request, as well as the log recording level and log dump level of the log recording module object, specifically includes:

[0064] Get the log level of the current log recording point, and compare the current log level of the current log recording point with the log recording level of the log recording module object.

[0065] If the log level of the current log recording point is greater than or equal to the log recording level of the log recording module object, then the processing operation log data of the client operation request of the previous log recording point is recorded in the log storage buffer of the log recording module object;

[0066] If the log level of the current log recording point is lower than the log recording level of the log recording module object, then the processing operation log of the client operation request of the previous log recording point will not be recorded.

[0067] in,

[0068] Log points are recording points set up to record or output the processing operation logs of client operation requests. Each log point has a corresponding log level set in advance. Log points are set according to the client connection object, server connection object, and the derived connection object corresponding to the client connection object. Through the log point, the client connection object, server connection object, or derived connection object corresponding to the log point can be obtained.

[0069] The log storage buffer of the log recording module object is a fixed-length memory space used to temporarily store the processing logs of client operation requests.

[0070] In this embodiment, step S4, the step of recording or outputting the log data of the current log recording point in real time according to the log level of the current log recording point processed by the operation request, and the log recording level and log dump level of the log recording module object, further includes a log dumping step, specifically:

[0071] Compare the current log level with the log dump level of the log recording module object.

[0072] If the log level of the current log recording point is greater than or equal to the log dump level of the log recording module object, then the processing operation log data of the client operation request stored in the log storage buffer of the log recording module object is stored in the disk, and the log storage buffer of the log recording module object is cleared.

[0073] If the log level of the current log recording point is less than or equal to the log dump level of the log recording module object, then the log is forced to be cached and dumped based on the amount of client operation request processing operation log data stored in the log storage buffer of the log recording module object.

[0074] In this embodiment, step S4, the step of performing forced log caching and dumping based on the amount of client operation request processing log data stored in the log storage buffer of the log recording module object, specifically includes:

[0075] If the amount of client operation request processing operation log data stored in the log storage buffer of the log recording module object is equal to the capacity of the log storage buffer of the log recording module object, then the client operation request processing operation log data stored in the log storage buffer of the log recording module object is stored in the disk, and the log storage buffer of the log recording module object is cleared.

[0076] To enable those skilled in the art to more accurately understand the above-mentioned technical solutions provided by the present invention, examples are provided to further supplement the explanation of the above-mentioned technical solutions.

[0077] refer to Figure 2 .

[0078] Step 1: Receive and parse the user's command to capture tracing logs. The command includes the log level, dump level, log buffer size, IP address mask, and port number of the connection object to be logged. From the parsed result, determine the client subnet address and port number of the session to be captured. Configure the log settings to set the subnet mask for logging. Here, "session" refers to the mapping between client and server connection objects established by a load balancing forwarding task. This mapping will persist for a period of time depending on the load balancing business, but the specific duration is irrelevant to this patent.

[0079] In this context, the client subnet address is the range of IP addresses assigned to each subnet within a network after subnetting, used to further differentiate and manage devices within the network. Alternatively, a subnet address can be defined as a division of the network portion of an IP address, used to further subdivide a large network into smaller networks. An IP address consists of two parts: the network address and the host address.

[0080] Step 2: The device listens to the network service port and waits for client connections. This step mainly involves load balancing logic and is not directly related to this patent, but it is retained for the sake of completeness.

[0081] Step 3: Receive a client connection request. The connection request refers to a SYN packet in the TCP protocol. This packet includes the client's source IP address and port number. Create a client connection object based on the content of the request packet. The source IP address and source port number of the request can be obtained through the client connection object later.

[0082] Based on the subnet mask and port number issued in step 1, and the source IP address field in the IP layer protocol header and the port number in the TCP layer protocol header of the connection request message, it is determined by calculating the subnet mask to determine if they belong to the same subnet. If the client's source IP address is within the subnet range described in the log capture instruction issued in step 1, then logs need to be recorded; otherwise, no logs need to be recorded, and the subsequent log processing logic is skipped, and the traffic is processed as normal traffic. The main task here is to narrow down the scope of log recording by matching the client's source IP address, port number, and the capture conditions described in the log capture instruction.

[0083] If the conditions are met, namely that the source IP address of the client connection request is within the set client subnet address and the port number is consistent, then proceed to step 4;

[0084] Step 4: If the client connection meets the filtering criteria—that is, the source IP address of the client connection request is within the set client subnet address and the port number is consistent—then a logging module object is constructed, and the logging module object pointer on the client connection object is set to that location. The logging module object is a structure comprising three main parts: logging level, log dump level, and log storage buffer.

[0085] Step 5: In subsequent steps, all log points will check if a logging module exists for the currently operating connection. If it does, the log level on the logging module object will be checked. If the log level of the log point is greater than or equal to the level described by the logging module object, the log will be recorded in the log cache on the logging module object. The log cache is a fixed-length memory space used to temporarily store low-level logs. When the log level of the current log point is greater than or equal to the log dump level described in the logging module object, the cached content on the logging module object will be stored on disk, and the cached content will be cleared. If the log dump level has not been reached when logging, but the log cache is full, a forced dump will be performed. Writing the cached content to disk in batches is done to reduce disk access frequency and improve disk storage performance.

[0086] Step 6: After the network device completes the load balancing logic, selects the forwarding target, and establishes the backend server connection object, the log module object pointer on the server connection object points to the log module object on the client connection object.

[0087] Step 7: In subsequent processes, the operations related to server object connection will also be logged in the manner described in Step 5.

[0088] By applying the above technical solutions, defining the log recording module object structure and log tracking scope, and obtaining the corresponding source IP address and port number based on the log tracking scope and client connection requests, a log recording module object is constructed for client connection requests. Based on the constructed log recording module object and the log level of the current log recording point, log data of the load balancing system's processing flow is recorded. This achieves selective recording and storage of log data of the load balancing system's processing flow without affecting the load balancing system's performance. Furthermore, it allows for rapid troubleshooting and location of log data related to the relevant links when the load balancing system malfunctions. This solves the technical problems of existing technologies, such as massive log recording volume, slowing down the load balancing system's processing capacity, and severe performance fluctuations that fail to provide effective support for fault diagnosis. Therefore, it reduces the amount of log data and system performance pressure, improves the accuracy of log recording and the efficiency of log-based fault diagnosis, and enhances the overall performance and stability of the load balancing system.

[0089] Furthermore, the technical solution provided by this invention effectively avoids recording related log information for all requests, reducing the pressure on the logging system. Log content is only recorded when a connection meets the conditions for logging. Simultaneously, the client and server connections use the same logging module object, and the content of this log is recorded in the same location, avoiding the problem of log interleaving between different requests.

[0090] Corresponding to the log tracing method based on a load balancing device described in one embodiment of the present invention, the present invention also discloses a log tracing device based on a load balancing device, such as... Figure 3 As shown, the device includes a log tracking range processing module, a client listening module, a log recording module object association module, and a log recording storage module;

[0091] in,

[0092] The log tracking range processing module is used to determine the log tracking range, define the log recording module object structure, receive and parse the tracking log capture command issued by the user in real time, and determine the log tracking range of the client connection object that needs to be tracked based on the parsing result of the tracking log capture command.

[0093] The client listening module is used for client listening. Based on the load balancing device, it listens to the network service port in real time, obtains the source IP address, port number and packet corresponding to the client connection request according to the listened client connection request, creates a client connection object according to the packet corresponding to the client connection request, determines the server-side connection object according to the load balancing policy provided by the load balancing device and the client connection object, and creates a session.

[0094] The log recording module object association module is used to generate and associate log recording module objects. Based on the log tracking range and the corresponding source IP address and port number obtained from the client connection request, the log recording module object is constructed. The client connection object, as well as the derived connection object and server connection object corresponding to the client connection object, are associated with the log recording module object through the log module object pointer, and the value of the log module object pointer is updated.

[0095] The log recording and storage module is used for log recording and storage. Based on the load balancing device and the session response, it processes the client's operation requests and records or outputs the log data of the current log recording point in real time according to the log level of the current log recording point processed by the operation request, as well as the log recording level and log dump level of the log recording module object.

[0096] in,

[0097] The log tracking scope includes the IP address subnet mask and port number.

[0098] The various embodiments in this specification are described in a related manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0099] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A log tracing method based on a load balancer, characterized in that, The method includes: S1, Determine the log tracking scope: Define the log recording module object structure, receive and parse the user's log tracking capture command in real time, and determine the log tracking scope of the client connection object that needs to be tracked based on the parsing result of the log tracking capture command; S2, Client Listening: Based on the load balancing device, the network service port is monitored in real time. The source IP address, port number and packet corresponding to the client connection request are obtained according to the monitored client connection request. A client connection object is created according to the packet corresponding to the client connection request. The server-side connection object is determined according to the load balancing policy provided by the load balancing device and the client connection object, and a session is created. S3, Log recording module object generation and association: Based on the log tracking range and the corresponding source IP address and port number obtained from the client connection request, a log recording module object is constructed, and the client connection object, as well as the derived connection object and server-side connection object corresponding to the client connection object, are associated with the log recording module object through the log module object pointer, and the value of the log module object pointer is updated. S4, Log recording and storage: Based on the load balancing device and the session response, process the client's operation request, and record or output the log data of the current log recording point in real time according to the log level of the current log recording point processed by the operation request, as well as the log recording level and log dump level of the log recording module object; in, The log tracking scope includes the IP address subnet mask and port number; In step S3, the step of associating the client connection object, the corresponding derived connection object, and the server connection object with the log recording module object through the log module object pointer further includes: When the IP address subnet mask of the client connection object that needs to be logged contains a client subnet address that is the same as the client subnet address corresponding to the client connection request, and the port number of the client connection object that needs to be logged contains a port number that is the same as the port number corresponding to the client connection request, the log module object pointer of the client connection object corresponding to the client connection request is set to point to the log recording module object, and the value of the log module object pointer of the client connection object is updated. Set the log module object pointer of the server-side connection object corresponding to the client connection request to the log recording module object, and update the value of the log module object pointer of the server-side connection object; If the client connection object has a derived connection object, the log module object pointer of the derived connection object of the client connection object is made to point to the log recording module object, and the value of the log module object pointer of the derived connection object of the client connection object is updated. in, The derived connection object corresponding to the client connection object is the connection object corresponding to the server connection object determined according to the load balancing strategy, or the connection object corresponding to the server connection object obtained by executing the script related to the load balancing strategy.

2. The method as described in claim 1, characterized in that, In step S1, the step of determining the log tracking range further includes: S11, Define the log recording module object structure: Define the log recording module object structure according to the content of the log recording, wherein the log recording module object structure includes: log recording level, log dump level and log storage buffer; S12, Determine the log tracking scope of the client connection: Based on the load balancing device, receive the tracking log capture command issued by the user in real time, parse the tracking log capture command, obtain the log recording level, log dump level, log storage buffer storage capacity, and IP address subnet mask and port number of the client connection object that needs to be logged in the tracking log capture command, and determine the client subnet address and port number of the client connection that needs to be logged based on the obtained IP address subnet mask and port.

3. The method as described in claim 1, characterized in that, In step S2, the client listening step includes: S21, Based on the load balancing device, the network service port is monitored in real time. When a client connection request is detected, the client connection request is received and parsed to obtain the source IP address and source port number in the SYN packet corresponding to the client connection request. The client subnet address and port number corresponding to the client connection request are determined based on the source IP address and source port number. S22, determine and create a client connection object based on the SYN packet corresponding to the client connection request, and initialize the log module object pointer of the client connection object; The server-side connection object is determined based on the destination IP address and destination port number corresponding to the client connection request, and the log module object pointer of the server-side connection object is initialized. S23. Based on the client connection object and server connection object corresponding to the client connection request, determine whether a session corresponding to the client connection request already exists. If a session corresponding to the client connection request exists, forward the client connection request based on the session corresponding to the client connection request and the load balancing strategy, and record the log. If no session corresponding to the client connection request exists, create a new session. in, The SYN packet includes the source IP address, source port number, destination IP address, destination port number, and hash function; The client connection object is used to acquire and store TCP 5-tuple data sent by the client. The TCP 5-tuple data includes source IP address, source port number, destination IP address, destination port number, and protocol type. The initial values ​​of the log module object pointer of the client connection object and the log module object pointer of the server connection object are both null. The session is used to record the correspondence between client connection objects and server connection objects. When performing load balancing task forwarding, the task forwarding between client connection objects and server connection objects can be quickly completed through the session.

4. The method as described in claim 1, characterized in that, In step S3, the step of constructing a log recording module object based on the log tracking range and the corresponding source IP address and port number obtained from the client connection request further includes: The client subnet address and port number corresponding to the client connection request are compared with the IP address, subnet mask, and port number of the client connection object that needs to be logged. S311 If the IP address subnet mask of the client connection object that needs to record logs contains a client subnet address that is the same as the client subnet address corresponding to the client connection request, and the port number of the client connection object that needs to record logs contains a port number that is the same as the port number corresponding to the client connection request, then according to the log recording module object structure, a log recording module object is created for the client connection request, and the log recording level and log dump level are determined. The log storage buffer corresponding to the client connection request is initialized according to the storage capacity of the log storage buffer. S312, if the IP address subnet mask of the client connection object that needs to be logged does not contain a client subnet address that is the same as the client subnet address corresponding to the client connection request, or if the port number of the client connection object that needs to be logged does not contain a port number that is the same as the port number corresponding to the client connection request, then the client connection request does not need to be logged, and the client connection request is forwarded to the server based on the load balancing device, load balancing policy and session, as well as the destination IP address and destination port number corresponding to the client connection request; In this context, the data types of the log recording level and log dump level of the log recording module object corresponding to the client connection request are both integers.

5. The method as described in claim 1, characterized in that, In step S4, the step of recording or outputting the log data of the current log recording point in real time according to the log level of the current log recording point processed by the operation request, as well as the log recording level and log dump level of the log recording module object, specifically includes: Get the log level of the current log recording point, and compare the current log level of the current log recording point with the log recording level of the log recording module object. If the log level of the current log recording point is greater than or equal to the log recording level of the log recording module object, then the processing operation log data of the client operation request of the previous log recording point is recorded in the log storage buffer of the log recording module object; If the log level of the current log recording point is lower than the log recording level of the log recording module object, then the processing operation log of the client operation request of the previous log recording point will not be recorded. in, Log points are recording points set up to record or output the processing operation logs of client operation requests. Each log point has a corresponding log level set in advance. Log points are set according to the client connection object, server connection object, and the derived connection object corresponding to the client connection object. The client connection object, server connection object, or derived connection object corresponding to the log point can be obtained through the log point. The log storage buffer of the log recording module object is a fixed-length memory space used to temporarily store the processing logs of client operation requests.

6. The method as described in claim 1, characterized in that, In step S4, the step of recording or outputting the log data of the current log recording point in real time according to the log level of the current log recording point processed by the operation request, as well as the log recording level and log dump level of the log recording module object, further includes a log dumping step, specifically: Compare the current log level with the log dump level of the log recording module object. If the log level of the current log recording point is greater than or equal to the log dump level of the log recording module object, then the processing operation log data of the client operation request stored in the log storage buffer of the log recording module object is stored in the disk, and the log storage buffer of the log recording module object is cleared. If the log level of the current log recording point is less than or equal to the log dump level of the log recording module object, then the log is forced to be cached and dumped based on the amount of client operation request processing operation log data stored in the log storage buffer of the log recording module object.

7. The method as described in claim 6, characterized in that, In step S4, the step of performing forced log caching and dumping based on the amount of client operation request processing log data stored in the log storage buffer of the log recording module object specifically includes: If the amount of client operation request processing operation log data stored in the log storage buffer of the log recording module object is equal to the capacity of the log storage buffer of the log recording module object, then the client operation request processing operation log data stored in the log storage buffer of the log recording module object is stored in the disk, and the log storage buffer of the log recording module object is cleared.

8. An apparatus for implementing the log tracing method based on a load balancing device as described in claim 1, characterized in that, The device includes a log tracking range processing module, a client listening module, a log recording module object association module, and a log recording storage module; in, The log tracking range processing module is used to determine the log tracking range, define the log recording module object structure, receive and parse the tracking log capture command issued by the user in real time, and determine the log tracking range of the client connection object that needs to be tracked based on the parsing result of the tracking log capture command. The client listening module is used for client listening. Based on the load balancing device, it listens to the network service port in real time, obtains the source IP address, port number and packet corresponding to the client connection request according to the listened client connection request, creates a client connection object according to the packet corresponding to the client connection request, determines the server-side connection object according to the load balancing policy provided by the load balancing device and the client connection object, and creates a session. The log recording module object association module is used to generate and associate log recording module objects. Based on the log tracking range and the corresponding source IP address and port number obtained from the client connection request, the log recording module object is constructed. The client connection object, as well as the derived connection object and server connection object corresponding to the client connection object, are associated with the log recording module object through the log module object pointer, and the value of the log module object pointer is updated. The log recording and storage module is used for log recording and storage. Based on the load balancing device and the session response, it processes the client's operation requests and records or outputs the log data of the current log recording point in real time according to the log level of the current log recording point processed by the operation request, as well as the log recording level and log dump level of the log recording module object. The log tracking scope includes the IP address subnet mask and port number.

Images