A method and system for accurately identifying interface performance issues by analyzing Nginx logs.

Abstract

This invention discloses a method for accurately identifying interface performance issues by analyzing Nginx logs, belonging to the field of internet technology. The method includes: obtaining request information for each interface from Nginx logs; the request information includes interface request parameters and interface response duration; calculating the average response duration of each interface based on the interface response duration of the request information; and analyzing the interface request parameters based on abnormal and normal information to obtain analysis results. This invention also discloses a system for accurately identifying interface performance issues by analyzing Nginx logs. This invention collects and analyzes Nginx logs to identify detailed information on requests that significantly exceed the average interface response time, and through comparative analysis of the input parameters of these requests, it accurately identifies the parameters causing the increased response time, thereby enabling better performance optimization of the interface.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, specifically to a method and system for accurately identifying interface performance problems by analyzing Nginx logs. Background Technology

[0002] Nginx is a very popular free and open-source web server software. Currently, a large number of web services use Nginx for reverse proxying and load balancing. Nginx logs can record information such as the URL of the client's API call to the web service, the API's return status code, the API request parameters, and the API response time. We can perform statistical analysis on this information to discover performance issues with the web service API.

[0003] Currently, we also collect and analyze Nginx logs to statistically analyze metrics such as call count and average response time for various system interfaces. This helps us identify interfaces with long response times and optimize their performance. However, in actual business scenarios, there may be situations where the average response time of an interface meets requirements, but under certain specific request parameters, the response time is much longer than the average. A common example is when a specific parameter is not included in the input parameters, causing the interface logic to fail to properly utilize the database index, resulting in a slower response. Current statistical methods cannot accurately identify these types of problems. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for accurately identifying interface performance problems by analyzing Nginx logs, based on precise problem parameters.

[0005] To address the aforementioned technical problems, this invention provides a method for accurately identifying interface performance issues by analyzing Nginx logs, comprising the following steps:

[0006] Obtain request information for each interface from the Nginx logs; the request information includes interface request parameters and interface response duration;

[0007] Calculate the average response time of each interface based on the response time of the request information;

[0008] Based on the average response time and a first preset threshold, extract abnormal information from the request information of each interface;

[0009] Based on the average response time and the second preset threshold, normal information is extracted from the request information of each interface.

[0010] The analysis results are obtained by analyzing the interface request parameters based on the abnormal and normal information.

[0011] Preferably, the first preset threshold is equal to twice the average response time.

[0012] Preferably, based on the average response time and a first preset threshold, abnormal information is extracted from the request information of each interface, specifically including the following steps:

[0013] Requests whose response time to the same interface is more than twice the average response time are treated as exceptions.

[0014] Preferably, the second preset threshold is 10% above or below the average response time.

[0015] Preferably, normal information is extracted from the request information of each interface based on the average response time and a second preset threshold, specifically including the following steps:

[0016] Requests whose response time to the same interface fluctuates within 10% of the average response time are considered normal.

[0017] Preferably, the analysis is performed based on the interface request parameters of abnormal and normal information to obtain the analysis results, specifically including the following steps:

[0018] Interface request parameters that appear in error messages but not in normal messages will be considered suspicious parameters.

[0019] The suspicious parameters were analyzed, and the analysis results were obtained.

[0020] Preferably, interface request parameters that appear in abnormal information but not in normal information are considered suspicious parameters, specifically including the following steps:

[0021] Parse the parameter names of each interface request parameter in all interface requests with all exception information, and use them as the first set;

[0022] Parse the parameter names of each interface request parameter in all normal information interface requests and use them as the second set;

[0023] Find the difference between the first set and the second set;

[0024] Based on the difference set, the suspicious parameters are obtained.

[0025] Preferably, obtaining request information for each interface from Nginx logs specifically includes the following steps:

[0026] Use a log collection service to obtain request information for each interface from Nginx logs.

[0027] Preferably, the parameter name is saved in jsonPath format.

[0028] This invention also provides a system for accurately identifying interface performance problems by analyzing Nginx logs, comprising:

[0029] The acquisition module is used to obtain request information for each interface from Nginx logs; the request information includes interface request parameters and interface response duration.

[0030] The average response time calculation module calculates the average response time of each interface based on the interface response time of the request information.

[0031] The anomaly information extraction module extracts anomaly information from the request information of each interface based on the average response time and a first preset threshold.

[0032] The normal information extraction module extracts normal information from the request information of each interface based on the average response time and a second preset threshold.

[0033] The analysis module uses interface request parameters based on abnormal and normal information to perform analysis and obtain analysis results.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0035] This invention collects and analyzes Nginx logs to identify detailed information about requests that far exceed the average response time of the interface. By comparing and analyzing the input parameters of these requests, it can accurately identify the parameters that cause the longer response time, thereby enabling better performance optimization of the interface. Attached Figure Description

[0036] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0037] Figure 1 This is a flowchart illustrating the process of obtaining the analysis result report in Example 1;

[0038] Figure 2 This is a flowchart illustrating a method for accurately identifying interface performance issues by analyzing Nginx logs, as described in this invention. Detailed Implementation

[0039] Numerous specific details are set forth in the following description to provide a full understanding of the invention. However, the invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0040] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0041] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0042] The present invention will now be described in further detail with reference to the accompanying drawings:

[0043] This invention provides a method for accurately identifying interface performance issues by analyzing Nginx logs, comprising the following steps:

[0044] Obtain request information for each interface from the Nginx logs; the request information includes interface request parameters and interface response duration;

[0045] Calculate the average response time of each interface based on the response time of the request information;

[0046] Based on the average response time and a first preset threshold, extract abnormal information from the request information of each interface;

[0047] Based on the average response time and the second preset threshold, normal information is extracted from the request information of each interface.

[0048] The analysis results are obtained by analyzing the interface request parameters based on the abnormal and normal information.

[0049] Preferably, the first preset threshold is equal to twice the average response time.

[0050] Preferably, based on the average response time and a first preset threshold, abnormal information is extracted from the request information of each interface, specifically including the following steps:

[0051] Requests whose response time to the same interface is more than twice the average response time are treated as exceptions.

[0052] Preferably, the second preset threshold is 10% above or below the average response time.

[0053] Preferably, normal information is extracted from the request information of each interface based on the average response time and a second preset threshold, specifically including the following steps:

[0054] Requests whose response time to the same interface fluctuates within 10% of the average response time are considered normal.

[0055] Preferably, the analysis is performed based on the interface request parameters of abnormal and normal information to obtain the analysis results, specifically including the following steps:

[0056] Interface request parameters that appear in error messages but not in normal messages will be considered suspicious parameters.

[0057] The suspicious parameters were analyzed, and the analysis results were obtained.

[0058] Preferably, interface request parameters that appear in abnormal information but not in normal information are considered suspicious parameters, specifically including the following steps:

[0059] Parse the parameter names of each interface request parameter in all interface requests with all exception information, and use them as the first set;

[0060] Parse the parameter names of each interface request parameter in all normal information interface requests and use them as the second set;

[0061] Find the difference between the first set and the second set;

[0062] Based on the difference set, the suspicious parameters are obtained.

[0063] Preferably, obtaining request information for each interface from Nginx logs specifically includes the following steps:

[0064] Use a log collection service to obtain request information for each interface from Nginx logs.

[0065] Preferably, the parameter name is saved in jsonPath format.

[0066] This invention also provides a system for accurately identifying interface performance problems by analyzing Nginx logs, comprising:

[0067] The acquisition module is used to obtain request information for each interface from Nginx logs; the request information includes interface request parameters and interface response duration.

[0068] The average response time calculation module calculates the average response time of each interface based on the interface response time of the request information.

[0069] The anomaly information extraction module extracts anomaly information from the request information of each interface based on the average response time and a first preset threshold.

[0070] The normal information extraction module extracts normal information from the request information of each interface based on the average response time and a second preset threshold.

[0071] The analysis module uses interface request parameters based on abnormal and normal information to perform analysis and obtain analysis results.

[0072] To better illustrate the technical effects of the present invention, the present invention provides the following specific embodiments to illustrate the above technical process:

[0073] Example 1: A method for accurately identifying interface performance issues by analyzing Nginx logs, such as... Figure 1 As shown, it includes the following steps:

[0074] (1) Use a log collection service to collect the request information of the interface from the Nginx logs into the database. The request information includes the interface URL, the interface return status code, the interface request parameters, the interface response time, etc.

[0075] (2) Calculate the average response time of each interface by grouping by interface, and save the statistical results into a map. The key of the map is the interface URL address, and the value is the average response time of the interface.

[0076] (3) Iterate through the map of average response time statistics, and query the database in turn the input parameter information (interface request parameters of exception information) of all interfaces whose response time is far greater than the average response time (the standard of far greater can be customized, such as twice the average response time). Then, parse and extract the parameter names in turn, and save the parameter names that have appeared in any request into a set set1 (the first set set) in a format similar to jsonPath.

[0077] (4) Query the database for interface request parameters (normal information interface request parameters) of the interface call whose current interface response time is close to the average response time (the standard of closeness can be customized, such as the average response time fluctuating by 10%), and parse and extract the parameter names. Save the parameter names that have appeared in all requests into a set set2 (the second set set) in a format similar to jsonPath.

[0078] (5) Find the difference between the two sets, get all the parameter names that are in set2 but not in set1 and save them to set3. The longer the interface response time may be due to the input parameters not being passed to these parameters in set3. You can then use this information to optimize the interface performance.

[0079] (6) After all interfaces have been statistically analyzed, output the analysis results report.

[0080] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules, units, or units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units, modules, or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0081] The units may or may not be physically separate. The components shown as units can be one or more physical units, meaning they can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0082] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0083] In particular, according to embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the methods of this invention. It should be noted that the computer-readable medium described above in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.

[0084] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0085] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions within the technical scope disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for accurately identifying interface performance issues by analyzing Nginx logs, characterized in that, Includes the following steps: Obtain request information for each interface from the Nginx logs; the request information includes interface request parameters and interface response duration; Calculate the average response time of each interface based on the response time of the request information; Based on the average response time and a first preset threshold, extract abnormal information from the request information of each interface; Based on the average response time and the second preset threshold, normal information is extracted from the request information of each interface. Parse the parameter names of each interface request parameter in all interface requests with all exception information, and use them as the first set; Parse the parameter names of each interface request parameter in all normal information interface requests and use them as the second set; Find the difference between the first set and the second set; Based on the difference set, the suspicious parameters are obtained; The suspicious parameters were analyzed, and the analysis results were obtained.

2. The method for accurately identifying interface performance problems by analyzing Nginx logs according to claim 1, characterized in that: The first preset threshold is equal to twice the average response time.

3. The method for accurately identifying interface performance problems by analyzing Nginx logs according to claim 2, characterized in that, Based on the average response time and a first preset threshold, exception information is extracted from the request information of each interface, specifically including the following steps: Requests whose response time to the same interface is more than twice the average response time are treated as exceptions.

4. The method for accurately identifying interface performance problems by analyzing Nginx logs according to claim 3, characterized in that, Based on the average response time and a second preset threshold, normal information is extracted from the request information of each interface, specifically including the following steps: Requests whose response time to the same interface fluctuates within 10% of the average response time are considered normal.

5. The method for accurately identifying interface performance problems by analyzing Nginx logs according to claim 1, characterized in that, Obtaining request information for each interface from Nginx logs involves the following steps: Use a log collection service to obtain request information for each interface from Nginx logs.

6. The method for accurately identifying interface performance problems by analyzing Nginx logs according to claim 1, characterized in that: The parameter name is saved in jsonPath format.

7. A system for accurately identifying interface performance problems by analyzing Nginx logs, used to implement the method for accurately identifying interface performance problems by analyzing Nginx logs as described in any one of claims 1-6, characterized in that, include: The retrieval module is used to retrieve request information for each interface from the Nginx logs; The request information includes interface request parameters and interface response duration; The average response time calculation module calculates the average response time of each interface based on the interface response time of the request information. The anomaly information extraction module extracts anomaly information from the request information of each interface based on the average response time and a first preset threshold. The normal information extraction module extracts normal information from the request information of each interface based on the average response time and a second preset threshold. The analysis module uses interface request parameters based on abnormal and normal information to perform analysis and obtain analysis results.

Images