Interface Call Log Analysis Method, Device, Equipment, Medium and Product

Abstract

The present application discloses an interface call log analysis method, device, equipment, medium and product, which relates to the technical field and is applied to microservices installed with an interface call log analysis toolkit. The method includes: intercepting service requests for the interfaces of the microservices based on filters in the interface call log analysis toolkit, and parsing the service information of the intercepted service requests; converting the service information into log data based on a preset log format; obtaining requirement information input from the outside, configuring a collection strategy for the log data according to the requirement information, storing the target log data in the log data in a preset database according to the configured collection strategy, and performing anomaly detection on a data set composed of multiple target log data in the database; if the anomaly detection result is that there is an anomaly, outputting a preset warning message. The present application greatly reduces the complexity of the deployment and configuration of the log analysis tool.

Description

Technical Field

[0001] This application relates to the field of computer software technology, and particularly to an interface call log analysis method, apparatus, device, medium and product. Background Art

[0002] In recent years, with the wide application of the microservices architecture, the complexity of software systems has been continuously increasing. In this context, ensuring efficient collaboration and fault troubleshooting between microservices has become particularly important. However, existing link tracing and log analysis tools often have some defects, such as complex deployment and configuration processes, limited real-time analysis capabilities, and insufficient warning and notification functions. For example, systems like Zipkin (a distributed tracing system) provide the collection and display of tracing data, but require separate deployment of a tracing server and configuration of corresponding tracing dependencies and code in each service, which may be relatively complex for users new to such systems. Summary of the Invention

[0003] The main purpose of this application is to provide an interface call log analysis method, apparatus, device, medium and product, aiming to solve the technical problem of high complexity in the deployment and configuration of log analysis tools.

[0004] To achieve the above object, this application proposes an interface call log analysis method, which is applied to a microservice installed with an interface call log analysis tool package. The interface call log analysis method includes:

[0005] Intercept service requests for the interfaces of the microservice based on filters in the interface call log analysis tool package, and parse the service information of the intercepted service requests. The service information is extracted from the service requests and is used to describe the requests themselves and their contexts, including request time, requester identity, request parameters, response time, responder identity, response content, and other relevant information. The other relevant information includes user authentication information, client IP address, and request source;

[0006] Convert the service information into log data based on a preset log format;

[0007] Obtain demand information input from the outside world, configure a collection strategy for the log data according to the demand information, and collect target log data in the log data according to the configured collection strategy. The collection strategy includes collection at time intervals, collection by log size, collection by request path, and collection by user identity. The collection strategy obtains collection strategy parameters input by the user through a visual interface and generates a corresponding collection strategy;

[0008] Store the target log data in a preset database, and determine a data set composed of multiple target log data in the database, and perform anomaly detection on the data set. Among them, the database includes at least one of a relational database and a NoSQL database. The step of storing the target log data in the preset database includes: if the number of service requests within a preset time period is less than a preset threshold, save the target log data in the form of a table to the relational database; if the number of service requests within a preset time period is greater than or equal to the preset threshold, save the target log data to the NoSQL database. The step of performing anomaly detection on the data set includes: dividing the data set into different data groups according to the interface, and taking each target log data in each data group as a data object. Among them, each interface corresponds to a data group. After the step of taking each target log data in each data group as a data object, it includes obtaining the identity of the requester of the data object, and performing user behavior analysis according to the identity of the requester. Among them, the user behavior analysis includes judging the user group according to the identity of the requester with the most occurrences in each data group, using machine learning or data mining algorithms to analyze the user group in each data group, and identifying the behavior pattern of the user group according to the analysis result. If the data object meets the preset anomaly condition, mark the data object as an abnormal log, and determine that the anomaly detection result is that there is an anomaly. Before the step of storing the target log data in the preset database, it includes: if the number of log entries is too large in a short time, temporarily store the log data with the number of log entries in a virtual space. If the log size of the log data in the virtual space exceeds the preset storage size, match the log data in the virtual space with a preset collection strategy to store the log data in the virtual space in the preset database according to the preset collection strategy;

[0009] If the anomaly detection result is that there is an anomaly, output a preset warning message.

[0010] In an embodiment, the step of collecting the target log data in the log data according to the configured collection strategy includes:

[0011] If the collection strategy is to collect at time intervals, determine a timing task corresponding to the collection strategy, and collect the target log data in the log data according to the timing task. Among them, the timing task includes collecting log data based on a preset time interval;

[0012] If the collection strategy is to collect according to the log size, detect whether the stored data volume of the log data is greater than a preset stored data volume threshold. If it is greater than the preset stored data volume threshold, collect the log data as the target log data;

[0013] If the collection strategy is to collect by log level, the log data that meets the preset log level is collected as target log data;

[0014] If the collection strategy is to collect by request path, the log data that meets the preset request path is collected as target log data;

[0015] If the collection strategy is to collect by user identity, the log data that meets the preset user identity is collected as target log data.

[0016] In one embodiment, after the step of taking each target log data in each data group as a data object, it includes:

[0017] For each data group, if the number of data objects within a preset time range is greater than a preset service request quantity threshold, it is determined that the data objects within the preset time range meet the preset abnormal condition; and / or,

[0018] Determine the data objects corresponding to the service requests in the failed state in the data group. If the number of data objects corresponding to the service requests in the failed state is greater than a preset failure quantity threshold, it is determined that the data objects corresponding to the service requests in the failed state meet the preset abnormal condition; and / or,

[0019] Determine the return parameters corresponding to each data object in the data group. If there is a target return parameter containing a preset specific character among the return parameters, it is determined that the data object corresponding to the target return parameter meets the preset abnormal condition, where the specific character is a preset character, and the target return parameter containing the specific character meets the preset abnormal condition.

[0020] In addition, to achieve the above object, the present application also proposes an interface call log analysis device, which is set in a microservice installed with an interface call log analysis toolkit. The interface call log analysis device includes:

[0021] A log collection module that intercepts service requests for the interfaces of the microservice based on a filter in the interface call log analysis toolkit and parses the service information of the intercepted service requests, where the service information is extracted from the service request and is used to describe the request itself and its context, including request time, requester identity, request parameters, response time, responder identity, response content, and other relevant information, and the other relevant information includes user authentication information, client IP address, and request source;

[0022] A log format module that converts the service information into log data based on a preset log format;

[0023] The log collection module obtains the required information input from the outside world, configures the collection strategy for the log data according to the required information, and collects the target log data in the log data according to the configured collection strategy. Among them, the collection strategy includes collection at time intervals, collection by log size, collection by request path, and collection by user identity. The collection strategy obtains the collection strategy parameters input by the user through a visual interface and generates the corresponding collection strategy. Before the step of storing the target log data in a preset database, if the number of log entries is too large in a short time, the log data with the number of log entries is temporarily stored in a virtual space. If the log size of the log data in the virtual space exceeds the preset storage size, the log data in the virtual space is matched with the preset collection strategy to store the log data in the virtual space in the preset database according to the preset collection strategy;

[0024] The log analysis module stores the target log data in a preset database, determines the data set composed of multiple target log data in the database, and performs anomaly detection on the data set. Among them, the database includes at least one of a relational database and a Nosql database. The step of storing the target log data in a preset database includes: if the number of service requests is less than the preset threshold within a preset time period, the target log data is saved in the relational database in the form of a table; if the number of service requests is greater than or equal to the preset threshold within a preset time period, the target log data is saved in the Nosql database; the step of performing anomaly detection on the data set includes dividing the data set into different data groups according to the interface, and taking each target log data in each data group as a data object. Among them, each interface corresponds to a data group; after the step of taking each target log data in each data group as a data object, it includes obtaining the identity of the requester of the data object and performing user behavior analysis according to the identity of the requester. Among them, the user behavior analysis includes judging the user group according to the identity of the requester with the most occurrences in each data group, using machine learning or data mining algorithms to analyze the user group in each data group, and identifying the behavior pattern of the user group according to the analysis result; if the data object meets the preset anomaly condition, the data object is marked as an abnormal log, and the anomaly detection result is determined to be abnormal; before the step of storing the target log data in a preset database, if the number of log entries is too large in a short time, the log data with the number of log entries is temporarily stored in a virtual space. If the log size of the log data in the virtual space exceeds the preset storage size, the log data in the virtual space is matched with the preset collection strategy to store the log data in the virtual space in the preset database according to the preset collection strategy;

[0025] The log warning module outputs a preset warning message if the abnormal detection result indicates an abnormality.

[0026] In addition, to achieve the above object, the present application further provides an interface call log analysis device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. The computer program is configured to implement the steps of the interface call log analysis method as described above.

[0027] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the interface call log analysis method as described above are implemented.

[0028] In addition, to achieve the above object, the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the interface call log analysis method as described above are implemented.

[0029] One or more technical solutions proposed by the present application have at least the following technical effects:

[0030] This application integrates the interface call log analysis method through an interface call log analysis toolkit, enabling each microservice to simply install the toolkit and automatically intercept and deeply analyze interface service requests without additional configuration or cumbersome operations. Specifically, the microservice installed with this toolkit can intercept service requests for all interfaces pointing to the microservice through a filter and parse the service information in these service requests, achieving comprehensive capture and detailed parsing of the service request content. Subsequently, the microservice converts the service information into log data according to a preset log format. Obtain the required information input from the outside world, configure the collection strategy for the log data according to the required information, and collect the target log data in the log data according to the configured collection strategy. The collection strategy includes at least one of collecting at time intervals, collecting by log size, collecting by log level, collecting by request path, and collecting by user identity. The collection strategy configured according to the required information meets the diverse log management requirements, enabling the system to efficiently screen out valuable log data for specific business scenarios or requirements. Store the target log data in a preset database, including at least one of a relational database and a NoSQL database, which can meet the requirements of different business scenarios. Furthermore, the system supports retrieving and summarizing the target log data from the database to form a comprehensive data set. By deeply analyzing these data sets, the system can identify potential performance bottlenecks and abnormal behaviors and send preset warning messages to users accordingly. This process not only helps developers quickly locate problems and optimize service performance but also greatly improves the security and stability of the system. In summary, this application greatly reduces the complexity of log analysis tool deployment and configuration through an integrated interface call log analysis toolkit, achieving automated collection, formatting, storage, and analysis of log data. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with this application and, together with the specification, are used to explain the principles of this application.

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or in the prior art, the following will briefly introduce the accompanying drawings required for describing the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0033] Figure 1 It is a schematic flowchart of the first embodiment of the interface call log analysis method of this application;

[0034] Figure 2 It is a schematic flowchart of the second embodiment of the interface call log analysis method of this application;

[0035] Figure 3 This is a schematic diagram of the scenario for the interface call log analysis method of this application;

[0036] Figure 4 This is a schematic diagram of the module structure of the interface call log analysis device according to an embodiment of this application;

[0037] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the interface call log analysis method according to an embodiment of this application.

[0038] The implementation, functional features, and advantages of this application will be further described in conjunction with embodiments and with reference to the accompanying drawings. Detailed implementation manners

[0039] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not used to limit this application.

[0040] To better understand the technical solutions of this application, the following will be described in detail in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0041] Due to some limitations of current log analysis tools, they usually require separate deployment of a tracing server and configuration of corresponding tracing dependencies and code in each service. For users who are new to such tools, the deployment and configuration processes may be relatively complex. In addition, currently, mainly the collection and display of tracing data are provided, but the real-time analysis and monitoring capabilities are relatively weak. Although real-time tracing data can be viewed, advanced real-time analysis functions are lacking. The warning and notification functions are also relatively limited, mainly relying on user-defined rules and scripts, lacking a built-in, out-of-the-box warning and notification system, which may require additional development and configuration by users. At the same time, some log analysis tools only use a single database for log storage, which reduces the adaptability of the log analysis tools and cannot meet the requirements of different business scenarios.

[0042] The interface call log analysis method provided in this embodiment simplifies the deployment process. Each microservice only needs to install the interface call log analysis toolkit integrated with the interface call log analysis method, without any other redundant operations, and there is no need to separately deploy a tracing service like Zipkin (an open-source distributed real-time data tracing system). This method has high configuration flexibility and supports using a configuration file to set parameters for log collection, such as whether to enable log collection, log format, log sending method, etc. This embodiment supports multiple relational databases and NoSQL (a general term for non-relational databases) databases, and can adapt to the needs of different business scenarios, enabling users to select a suitable database for log storage according to the actual situation. This embodiment provides rich collection strategies, including collection at time intervals, collection based on log size, collection based on log level, etc. Users can set to collect logs once a minute according to business requirements, or collect logs when the log size exceeds a specific threshold to improve the efficiency of log collection. In addition, this embodiment provides multiple warning strategies, including real-time monitoring and warning of online problems. For example, specific keywords or logs of a specific level can be set as warning objects, and a warning notification is sent when the conditions are met to promptly handle online problems. Some user-specific behaviors can also be collected, and developers can configure them according to their own business dimensions. For example, after a certain interface is requested a certain number of times by a certain user within a certain period of time, a notification is sent to the enterprise's intelligent platform for convenient subsequent business analysis. This embodiment is mainly for link tracing of interfaces, and we can perform some intelligent analysis through interface logs; in the architecture design of log warning and collection strategies, we have fully considered various business scenarios and basically support all the scenarios required in the research and development; the intelligent analysis module is quite practical for some consumer-oriented systems.

[0043] It should be noted that the execution entity of this embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a terminal system, etc. that can implement the above functions. Hereinafter, a microservice system installed with the interface call log analysis toolkit will be used as an example to illustrate this embodiment and the following embodiments.

[0044] Based on this, this embodiment provides an interface call log analysis method, referring to Figure 1 , Figure 1 is a schematic flowchart of the first embodiment of the interface call log analysis method of this application.

[0045] In this embodiment, it is applied to a microservice installed with the interface call log analysis toolkit.

[0046] The interface call log analysis toolkit refers to a software package that integrates interface call log analysis methods and is deployed in a microservices environment. It is used to automatically intercept, parse, collect, store, and analyze the log data of interface calls. Microservices only need to install the toolkit to record and analyze the log of the microservice interface call chain, without any other redundant operations.

[0047] The interface call log analysis method includes steps S10 - S50:

[0048] Step S10, based on the filter in the interface call log analysis toolkit, intercept the service requests for the interfaces of the requesting microservices, and parse the service information of the intercepted service requests;

[0049] It should be noted that an interface is a bridge for interaction between different components in a software system, defining the ways and rules for communication between components. In a microservices architecture, a service request refers to a request initiated by one microservice to another microservice or an external system, usually containing information such as the request method, path, parameters, etc. Service information is extracted from the service request and is used to describe the request itself and its context, including the request information and response information of the request. The request information can include the request time, the identity of the requester, request parameters, etc., and the response information can include the response time, the identity of the responder, response content, etc. The filter in the interface call log analysis toolkit can intercept these requests and responses before they reach the target resource or before returning from the target resource and process them.

[0050] The interface call log analysis toolkit has a built-in service request filter that can automatically intercept all service requests and responses without manually configuring tracing code or separately deploying a tracing server. When the microservices system installed with this toolkit receives external or internal service requests, the system first intercepts these service requests and extracts key service information from the intercepted service requests, such as the request time, request method, request path, request parameters, etc.

[0051] In addition, according to specific business requirements and preset configurations, other relevant information may also be extracted, such as user authentication information in the request headers (Headers), client IP address, request source, etc., for more comprehensive log analysis and security auditing.

[0052] Step S20, convert the service information into log data based on a preset log format;

[0053] It should be noted that the format of service information is not necessarily unified, so the original service information needs to be sorted according to a certain format for subsequent storage, query, and analysis.

[0054] The parsed service information is sorted and converted according to a preset format (such as JSON, a widely adopted lightweight data exchange format) to form log entries that are easy to store and query, and the converted log entries are used as log data.

[0055] Step S30: Obtain the requirement information input from the outside world, configure the collection policy for the log data according to the requirement information, and collect the target log data in the log data according to the configured collection policy. The collection policy includes at least one of collecting at time intervals, collecting by log size, collecting by log level, collecting by request path, and collecting by user identity.

[0056] It should be noted that the collection policy refers to the rule for collecting log data according to specific conditions (such as time interval, log size, log level, request path, user identity, etc.).

[0057] Obtain the requirement information for log collection from the user or system configuration, configure the corresponding collection policy according to the requirement information. The specific application of the collection policy can include collecting at time intervals, such as collecting logs at fixed time intervals like daily or hourly; collecting by log size, collecting when the log file reaches a certain size (such as 100MB); collecting by log level, only collecting logs of specific levels (such as ERROR, WARN); collecting by request path, only collecting logs of specific API paths; collecting by user identity, screening logs according to the user identity of the requester (such as user ID, role). According to the configured collection policy, filter out the target log data that meets the collection policy from the log data for collection.

[0058] Furthermore, the user can input the collection policy through a visual interface or a configuration file. For example, after the user logs in to the system, through the options and input boxes on the visual interface, fill in or select the required collection policy parameters. The system receives and parses these inputs to generate the corresponding collection policy; or the user edits a configuration file, writes the required collection policy parameters into the file. Then, the user uploads the configuration file to the system or places it in the directory specified by the system. The system reads the configuration file, parses the content therein, and generates the collection policy. Thus, the system filters out the target log data that meets the collection policy from the log data for collection.

[0059] Step S40: Store the target log data in a preset database, determine the data set composed of multiple target log data in the database, and perform anomaly detection on the data set. The database includes at least one of a relational database and a NoSQL database.

[0060] It should be noted that a database is a software system used to store and manage data, including relational databases (such as MySQL) and NoSQL databases. A relational database is a database system that uses tables to store and organize data. The data is stored in the form of rows and columns, and it supports complex queries and transaction processing. NoSQL (Not Only SQL) databases are a general term for a class of non-relational databases. They do not follow the rules of traditional relational databases, such as table structure, data relationships (such as foreign key constraints), and SQL query language. The original intention of NoSQL databases is to solve the problems of diversification and high-concurrency access of large-scale data sets. These characteristics make them perform well in dealing with big data, high availability, distributed systems, etc. Anomaly detection refers to analyzing the stored data set to identify data that does not conform to the normal pattern or expectation.

[0061] Store the collected target log data in a preset database. In the database, combine multiple target log data into a data set. Then analyze the data set. If the data set is small in scale and the analysis task is relatively simple, it can be directly analyzed in the database using the database's query language and built-in analysis functions. If the data set is huge in scale or complex data analysis and visualization are required, the data can be extracted and analyzed in an external environment (such as using programming languages like Python, R and their data analysis libraries). Analyze whether there is abnormal data, such as an abnormally long response time, a high request failure rate, etc.

[0062] Step S50, if the anomaly detection result is that there is an anomaly, output a preset warning message.

[0063] It should be noted that a warning message refers to a notice or alarm sent to relevant personnel when the system detects an abnormal situation, so as to take measures to solve the problem in a timely manner.

[0064] Based on the results of anomaly detection, determine whether there are anomalies, such as: performance anomalies (abnormally long response time or decreased processing speed), error rate anomalies (high request failure rate or increased internal system errors), resource usage anomalies (excessively high memory usage or insufficient disk space). If there are anomalies, generate warning messages according to the preset warning templates, which can include the following elements: anomaly type, occurrence time, affected scope, severity level, and recommended measures. Then send the warning messages to relevant personnel via email, SMS, instant messaging, etc. If there are no anomalies, continue monitoring and perform subsequent processing according to the configured policies (such as regular reports, log archiving, etc.). In some cases, the system may record the current state as a normal baseline for future anomaly detection comparison. In addition, for a stable state with no anomalies for a long time, the system may trigger some optimization or cleanup tasks, such as compressing old logs, releasing unused resources, etc., to maintain the best performance of the system.

[0065] Exemplarily, assume there is an e-commerce microservice system that includes a microservice for processing orders. To monitor the performance and stability of the order processing interface, we deploy an interface call log analysis toolkit. When the order processing interface receives a request to create an order, the interface call log analysis toolkit first intercepts this request and extracts key information from the request, such as the request time (2023-04-01 12:00:00), request method (POST), request path ( / orders), and request parameters ({userId: 123, productId: 456, quantity: 2}). Format these information into a JSON-formatted log entry: {"time": "2023-04-01 12:00:00", "method": "POST", "path": " / orders", "params": {"userId": 123, "productId": 456, "quantity":2}}. According to the preset collection policy (such as collecting once a day), save the formatted log data to the database. Retrieve the order processing interface log data for the most recent week from the database to form a dataset. Conduct statistical analysis on the dataset and find that the interface response time has increased significantly during a certain period, and the average response time has increased from 100ms to 300ms. According to the analysis results, send a warning message to the system administrator: "Attention! The response time of the order processing interface has increased significantly during the XX period, which may affect the user experience. Please check and optimize."

[0066] In this embodiment, an interface call log analysis method is integrated through an interface call log analysis toolkit, enabling each microservice to simply install the toolkit without additional configuration or cumbersome operations to automatically intercept and deeply analyze interface service requests. Specifically, the microservice installed with this toolkit can intercept service requests for all interfaces pointing to the microservice through a filter and parse the service information in these service requests, achieving comprehensive capture and detailed parsing of the service request content. Subsequently, the microservice converts the service information into log data according to a preset log format. Obtain the required information input from the outside world, configure the collection policy for the log data according to the required information, and collect the target log data in the log data according to the configured collection policy, where the collection policy includes at least one of collecting at time intervals, collecting by log size, collecting by log level, collecting by request path, and collecting by user identity. The collection policy configured according to the required information meets the diverse log management requirements, enabling the system to efficiently screen out valuable log data for specific business scenarios or requirements. Store the target log data in a preset database, including at least one of a relational database and a NoSQL database, which can meet the requirements of different business scenarios. Furthermore, the system supports retrieving and summarizing the target log data from the database to form a comprehensive data set. By deeply analyzing these data sets, the system can identify potential performance bottlenecks and abnormal behaviors and send preset warning messages to users accordingly. This process not only helps developers quickly locate problems and optimize service performance but also greatly improves the security and stability of the system. In summary, this embodiment greatly reduces the complexity of log analysis tool deployment and configuration through an integrated interface call log analysis toolkit, realizing the automatic collection, formatting, storage, and analysis of log data.

[0067] In a feasible implementation manner, the step of storing the target log data in a preset database in step S40 may include steps T10 to T20:

[0068] Step T10, if the number of service requests within a preset time period is less than a preset threshold, save the target log data to a relational database;

[0069] It should be noted that in this embodiment, various popular relational databases and NoSQL databases can be supported to save logs. The preset threshold is a predefined numerical boundary used to determine whether a certain condition is met. In this embodiment, the preset threshold is used to determine whether the number of service requests has reached a certain level, thereby determining the storage location of the log data.

[0070] Judgment is made manually based on the service request volume received by the microservice per unit time to determine whether it is small enough. If it is small enough, for example, only 10 requests are received in one minute, then only a relational database needs to be used for subsequent log storage, with lower costs.

[0071] Step T20: If the number of service requests is greater than or equal to the preset threshold within the preset time period, save the target log data to the NoSQL database.

[0072] Judgment is made manually based on the service request volume received by the microservice per unit time to determine whether it is large enough. If it is large enough, for example, 10,000 requests are received in one minute, then a NoSQL database needs to be used for subsequent log storage to meet the storage requirements of high concurrency and massive data.

[0073] Furthermore, the judgment of whether the number of service requests is less than the preset threshold can be achieved by writing code logic. The code will monitor or count the number of service requests and compare it with the preset threshold to make a decision. First, the system needs to count the number of service requests received within a period of time. This number can be obtained through a counter, the backlog of the message queue, or other mechanisms. Compare the counted number of service requests with the preset threshold to determine the storage location of the log data. If the number of service requests is less than the preset threshold: The system believes that the current request volume is small, and the high concurrency and scalability advantages provided by the NoSQL database may not be needed. Therefore, save the log data to the relational database to utilize the advantages of the relational database in transaction processing, complex queries, etc. The system believes that the current request volume is large, and a NoSQL database may be needed to handle the storage requirements of high concurrency and massive data. Therefore, save the log data to the NoSQL database to better handle large-scale data and high-concurrency access.

[0074] This embodiment significantly improves the flexibility and efficiency of log processing by selecting a data storage scheme. Decide the storage location of the log data according to the service request volume. When the request volume is small, utilize the low cost and high consistency advantages of the relational database to effectively reduce the storage cost; when the request volume is large, use the NoSQL database to cope with the challenges of high concurrency and massive data, ensuring the integrity of the log data and the stability of the system. This data storage strategy that autonomously selects based on the request volume not only improves the resource utilization efficiency but also enhances the scalability and response ability of the system, bringing significant beneficial effects to log management under the microservice architecture.

[0075] Based on Embodiment 1 of this application, in Embodiment 2 of this application, the same or similar content as that in the above Embodiment 1 can be referred to the above introduction and will not be repeated hereinafter. On this basis, refer to Figure 2 ,Figure 2 This is a schematic flowchart of the second embodiment of the method for analyzing interface call logs of the present application. The step of collecting target log data from the log data according to the configured collection policy in step S30 further includes steps A10 to A50:

[0076] Step A10, if the collection policy is to collect at time intervals, determine the scheduled task corresponding to the collection policy, and collect the target log data from the log data according to the scheduled task, where the scheduled task includes collecting log data based on a preset time interval;

[0077] It should be noted that collecting at time intervals is a collection policy, which means collecting log data according to a preset time interval (such as every hour, every day, etc.). A scheduled task is a task preset in a system or program that automatically executes according to a specific time or condition. In this embodiment, the scheduled task is used to collect log data based on a time interval. The target log data is the specific log data selected and ready to be collected according to the collection policy. The preset time interval is the time period set in advance in the collection policy of collecting at time intervals to determine the frequency of collecting log data.

[0078] Determine that the collection policy is to collect at time intervals, find the scheduled task corresponding to this policy, and automatically execute the collection operation of log data according to the preset time interval in the scheduled task.

[0079] Step A20, if the collection policy is to collect according to the log size, detect whether the stored data volume of the log data is greater than the preset stored data volume threshold. If it is greater than the preset stored data volume threshold, collect the log data as the target log data;

[0080] It should be noted that collecting according to the log size is a collection policy, which means that when the stored data volume of the log data reaches or exceeds a certain preset threshold, the collection of the log data is triggered. The preset stored data volume threshold is the boundary of the stored data volume used to determine whether to collect log data in the collection policy of collecting according to the log size.

[0081] Determine that the collection policy is to collect according to the log size, regularly detect the stored data volume of the log data, and if the stored data volume is greater than the preset stored data volume threshold, trigger the collection operation of the log data.

[0082] Step A30, if the collection policy is to collect according to the log level, collect the log data that meets the preset log level as the target log data;

[0083] It should be noted that collecting by log level is a collection strategy, which means collecting only log data with specific log levels. Log levels are usually used to indicate the importance and urgency of log information. The preset level refers to the specified log level to be collected in the collection strategy of collecting by log level (such as INFO, WARNING, ERROR, etc.).

[0084] Determine that the collection strategy is to collect by log level, traverse the log data, check the level of each log, and collect all log data that meets the preset level.

[0085] Step A40, if the collection strategy is to collect by request path, then collect the log data that meets the preset request path as the target log data.

[0086] It should be noted that collecting by request path is a collection strategy, which means collecting only log data related to specific request paths. This helps to analyze the logs of specific functions or modules. The preset path is the specified request path for which logs need to be collected in the collection strategy of collecting by request path.

[0087] For each log, its request path field needs to be checked. Compare the request path of the log with all preset request paths to see if there is a match. If the request path of the log matches a preset request path, then collect the log data for subsequent processing.

[0088] Step A50, if the collection strategy is to collect by user identity, then collect the log data that meets the preset user identity as the target log data.

[0089] It should be noted that collecting by user identity is a collection strategy, which means collecting only log data of specific user identities. This helps to track and analyze the operation behaviors of specific users. The preset identity is the specified user identity for which logs need to be collected in the collection strategy of collecting by user identity.

[0090] Determine that the collection strategy is to collect by user identity, check the user identity identifier in the log data, and collect all log data with the preset identity identifier.

[0091] Furthermore, multiple preset collection strategies can exist simultaneously. For example, if the preset collection strategies are to collect by request path (preset request path: / users / info) and by log level (preset level: ERROR), then a log with a request path of / users / info and a log level of ERROR will be collected. In addition, when the number of logs is excessive in a short period of time, a virtual space can be opened for temporary storage. When the size of the logs in the virtual space exceeds the preset storage size, then take them out, match them with the preset collection strategy, and save them to the database. It is also possible to take logs from the virtual space regularly for operations.

[0092] Through the above diversified log collection strategies, this embodiment can significantly improve the efficiency and accuracy of log management. Through flexible configuration, the system can automatically and accurately screen and collect target log data according to conditions such as time interval, log size, log level, request path, or user identity. This not only helps to quickly locate problems and analyze system performance but also effectively reduces the storage of unnecessary logs and lowers the storage cost. At the same time, when the log volume surges, virtual space is used for temporary storage, and then batch processing is carried out in conjunction with the preset collection strategy to ensure the continuity and efficiency of log processing, providing strong support for the stable operation of the system.

[0093] Based on Embodiment 1 or Embodiment 2 of the present application, in Embodiment 3 of the present application, for the same or similar content as in Embodiment 1 or Embodiment 2 above, reference can be made to the above introduction and will not be elaborated hereinafter. The step of saving the log data to the database in step S40 further includes steps B10 to B20:

[0094] Step B10, divide the data set into different data groups according to the interfaces, and use each target log data in each data group as a data object; wherein, each interface corresponds to a data group;

[0095] It should be noted that the data set is a collection of related data, which can be structured or unstructured and is used for analysis, processing, or storage. The data group is a subset obtained by dividing the data set according to the interfaces. The data object refers to each target log data in the data group, which is processed or analyzed individually.

[0096] First, according to the different interfaces, the data set is divided into multiple different data groups. Each interface corresponds to a unique data group, ensuring clear logic and easy management of the data. In each data group, each target log data is extracted as a data object. These data objects will serve as the basic units for subsequent analysis or processing.

[0097] Step B20, if the data object meets the preset abnormal conditions, mark the data object as an abnormal log and determine that the abnormal detection result is that there is an abnormality.

[0098] It should be noted that the preset abnormal conditions are a set of rules or conditions defined in advance for judging whether a data object is abnormal. When the attributes or values of the data object match these conditions, the data object can be considered abnormal. The abnormal log is the data object marked as abnormal, which may contain errors, abnormal behaviors, or data that does not meet expectations. The abnormal detection result is the conclusion obtained after performing abnormal detection on the data set, indicating whether there is abnormal data in the data set.

[0099] Traverse the data objects in each data group and apply the preset exception conditions for judgment. If a certain attribute or value of a data object meets the exception conditions, mark the data object as an exception log. After completing the exception detection of all data objects, determine the exception detection result according to the number of data objects marked as exception logs or other relevant factors. If there are data objects marked as exceptions, the exception detection result is that there are exceptions; otherwise, the result is that there are no exceptions.

[0100] Through this embodiment, the system can efficiently divide a large-scale data set into multiple data groups according to interfaces, achieving refined management and analysis of data. The data objects within each data group are used as independent units, facilitating subsequent in-depth analysis and processing. At the same time, the introduction of preset exception conditions enables the system to automatically identify and mark exception logs, quickly locate potential problems, and improve the accuracy and efficiency of exception detection. This process not only optimizes the log processing process but also enhances the stability and reliability of the system, providing strong support for users.

[0101] Based on any one of Embodiments 1 to 3 of the present application, in Embodiment 4 of the present application, the same or similar content as any one of the above Embodiments 1 to 3 can be referred to the above introduction and will not be elaborated hereinafter. After the step of taking each target log data in each data group as a data object in step B10, steps C10 to C30 are further included:

[0102] Step C10, for each data group, if the number of data objects within a preset time range is greater than a preset service request quantity threshold, determine that the data objects within the preset time range meet the preset exception conditions;

[0103] It should be noted that the preset time range is a pre-defined time interval for analyzing or screening data. The data objects within this time range will be particularly concerned or processed. The preset service request quantity threshold is a preset value used to determine whether the number of service requests within a specific time range is abnormal. If the actual quantity exceeds this threshold, it is considered that there is an abnormal situation.

[0104] For each data group, first determine a preset time range and count the number of data objects within this time range. Compare the statistical result with the preset service request quantity threshold. If the quantity is greater than the threshold, determine that the data objects within this time range meet the preset exception conditions.

[0105] Step C20, determine the data objects corresponding to the service requests in the failed state in the data group. If the number of data objects corresponding to the service requests in the failed state is greater than a preset failure quantity threshold, determine that the data objects corresponding to the service requests in the failed state meet the preset exception conditions;

[0106] It should be noted that a service request in a failure state refers to a service request that fails to be successfully completed, usually caused by system errors, network problems, resource shortages, etc. The preset failure quantity threshold is a preset value used to determine whether the number of service requests in a failure state in a data group is abnormal. If the actual quantity exceeds this threshold, it is considered that there is an abnormal situation.

[0107] Traverse each data object in the data group to identify the data objects corresponding to the service requests in a failure state. Count the number of these data objects and compare the statistical result with the preset failure quantity threshold. If the quantity is greater than the threshold, it is determined that the data objects corresponding to the service requests in a failure state meet the preset abnormal conditions.

[0108] Step C30: Determine the return parameters corresponding to each data object in the data group. If there is a target return parameter containing a preset specific character among the return parameters, it is determined that the data object corresponding to the target return parameter meets the preset abnormal conditions.

[0109] It should be noted that the return parameter is the data or information returned by the service provider to the requester after the service request is completed. These parameters usually contain information about the service execution result or status. The preset specific character is a specific character preset in data processing, which is used to identify or filter data with specific meanings or attributes. If the return parameter contains these characters, it may indicate that the data object meets a certain abnormal condition. The target return parameter is those parameters in the return parameter that contain the preset specific character, and the data objects corresponding to these parameters may meet the preset abnormal conditions.

[0110] Traverse each data object in the data group to obtain its corresponding return parameter. Check whether each return parameter contains the preset specific character. If a return parameter containing the specific character (i.e., the target return parameter) is found, it is determined that the data object corresponding to the return parameter meets the preset abnormal conditions.

[0111] Of course, there are more abnormal situations than those mentioned above, and users can also increase the detection of abnormal situations through configuration. At the same time, the system can also perform performance analysis, such as evaluating the interface performance according to indicators such as interface call time and response status code.

[0112] This embodiment significantly improves the speed of problem discovery and response by detecting anomalies in data groups in multiple dimensions. The detection of the number of data objects within a preset time range, the monitoring of the number of failed service requests, and the screening of specific return parameters together build a comprehensive and flexible anomaly detection framework. This not only helps users quickly locate potential problems, but also timely analyzes service performance and interface stability, providing strong support for system optimization. In addition, users can customize anomaly detection rules, which further enhances the flexibility and adaptability of the system and ensures stable operation of the system.

[0113] In a feasible implementation manner, after the step of taking each target log data in each data group as a data object in step B10, steps D10 to D20 are also included:

[0114] Step D10, obtaining the identity of the requester of the data object;

[0115] It should be noted that the requester identity refers to the identity of the entity that initiates a data request or performs an operation. In logs or service request records, the requester identity is usually used to identify which user or system component initiated the request.

[0116] Extract the requester identity information of each data object (such as log records, service request records) from the data set. Ensure the accuracy and completeness of the requester identity information for subsequent analysis.

[0117] Step D20 , performing user behavior analysis based on the requester identity, wherein the user behavior analysis includes determining the user group based on the requester identity that appears most frequently in each data group.

[0118] It should be noted that user analysis is a data analysis method that aims to gain an in-depth understanding of user behavior, preferences, needs and other information by collecting and processing user data, so as to provide a basis for product optimization, market strategy formulation, etc. A user group is a collection of users with common characteristics or behavior patterns. In user analysis, user groups are usually divided according to certain attributes of users (such as age, gender, region, interests, etc.) or behaviors (such as purchasing habits, frequency of use, etc.). In this embodiment, the division is mainly based on the frequency of use of the analysis interface.

[0119] The extracted requester identity information is classified and organized for statistical analysis. The number of occurrences of each requester identity in each data group is counted to understand the level of activity of different users or user groups in the data group. Based on the requester identity with the most occurrences in each data group, the main user group of the data group is determined. This helps to identify which users or user groups are the main contributors or users of the data group.

[0120] Furthermore, the advantages of link tracing for interfaces can be utilized to perform some intelligent analysis through interface logs. For example, through machine learning or data mining algorithms, valuable information can be extracted from a large amount of logs to provide support for business optimization.

[0121] In this embodiment, by accurately obtaining and analyzing the requester identity of data objects, the main user groups and their behavior patterns are identified. This not only provides a strong basis for product optimization and market strategy formulation, but also enhances the depth and breadth of user analysis. Combining link tracing and intelligent analysis technologies, the system can deeply mine valuable information in interface logs, provide more accurate data support for business decisions, and further improve business efficiency and user satisfaction.

[0122] It should be noted that in the case where the technology is feasible and the logic is clear, the above embodiments can be combined in pairs or freely combined in multiple ways.

[0123] Exemplarily, referring to Figure 3 , Figure 3 is a flowchart of the interface call log analysis method of this application. Suppose there is a large e-commerce platform whose microservice architecture includes multiple core services such as order service, inventory service, and payment service. The interface call links between these services are complex. To ensure the stability of the system and the user experience, the platform decides to use this application to record and analyze the interface call link logs. The platform first initializes the system, that is, introduces the tool package provided by the present invention into each microservice. After the tool package captures each service request, it extracts key information (such as request path, request parameters, response status code, response content, etc.), and formats it. Then, according to the business scale and data volume of the platform, Elasticsearch (a highly scalable distributed full-text search engine) is selected as the log storage database to ensure efficient log retrieval and analysis capabilities. After saving, the system can visually display the logs, and users or developers can intuitively view the stored log data. It is also possible to view the call links, call times, and response statuses of each service intuitively through the interface query method. To further optimize the user experience and improve business efficiency, the platform also introduces the intelligent analysis of this application. By collecting and analyzing user access logs, the products and user groups with high-frequency purchases are identified, providing strong support for subsequent personalized recommendations and promotional activities. More importantly, the intelligent analysis can identify abnormal situations by analyzing log data, such as interface call failures, too long response times, etc. If an abnormal situation is identified by the intelligent analysis, the intelligent analysis will notify the warning monitoring to send optimization suggestions to the user. If there is no abnormal situation, this log analysis will end.

[0124] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the method for analyzing interface call logs of this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0125] This application also provides an interface call log analysis device. Please refer to Figure 4 , which is set in the microservice installed with the interface call log analysis toolkit. The interface call log analysis device includes:

[0126] A log collection module 10 intercepts service requests for the interfaces of the microservice based on filters in the interface call log analysis toolkit, and parses the service information of the intercepted service requests. The service information is extracted from the service request and is used to describe the request itself and its context, including request time, requester identity, request parameters, response time, responder identity, response content, and other relevant information. The other relevant information includes user authentication information, client IP address, and request source;

[0127] A log format module 20 converts the service information into log data based on a preset log format;

[0128] A log collection module 30 obtains the requirement information input from the outside, configures the collection strategy for the log data according to the requirement information, and collects the target log data in the log data according to the configured collection strategy. The collection strategy includes collection at time intervals, collection by log size, collection by request path, and collection by user identity. The collection strategy obtains the collection strategy parameters input by the user through a visual interface and generates the corresponding collection strategy. Before the step of storing the target log data in a preset database, if the number of logs is too large in a short time, the log data with the number of logs is temporarily stored in a virtual space. If the log size of the log data in the virtual space exceeds the preset storage size, the log data in the virtual space is matched with the preset collection strategy to store the log data in the virtual space in the preset database according to the preset collection strategy;

[0129] The log analysis module 40 stores the target log data in a preset database, determines a data set composed of multiple target log data in the database, and performs anomaly detection on the data set. Among them, the database includes at least one of a relational database and a NoSQL database. The step of storing the target log data in a preset database includes: if the number of service requests within a preset time period is less than a preset threshold, saving the target log data in the form of a table to a relational database; if the number of service requests within a preset time period is greater than or equal to the preset threshold, saving the target log data to a NoSQL database. The step of performing anomaly detection on the data set includes dividing the data set into different data groups according to interfaces, and taking each target log data in each data group as a data object. Among them, each interface corresponds to a data group. After the step of taking each target log data in each data group as a data object, it includes obtaining the identity of the requester of the data object and performing user behavior analysis according to the identity of the requester. Among them, the user behavior analysis includes judging the user group according to the identity of the requester with the most occurrences in each data group, using machine learning or data mining algorithms to analyze the user group in each data group, and identifying the behavior pattern of the user group according to the analysis result. If the data object meets the preset anomaly condition, the data object is marked as an abnormal log, and the anomaly detection result is determined to be abnormal. Before storing the target log data in a preset database, it includes: if the number of logs is too large in a short time, temporarily storing the log data with the number of logs in a virtual space. If the log size of the log data in the virtual space exceeds the preset storage size, matching the log data in the virtual space with a preset collection policy to store the log data in the virtual space in a preset database according to the preset collection policy;

[0130] The log warning module 50 outputs a preset warning message if the anomaly detection result is abnormal.

[0131] The interface call log analysis device provided by this application adopts the interface call log analysis method in the above embodiment, which can solve the technical problem of high complexity in the deployment and configuration of log analysis tools. Compared with the prior art, the beneficial effects of the interface call log analysis device provided by this application are the same as those of the interface call log analysis method provided by the above embodiment, and other technical features in the interface call log analysis device are the same as the features disclosed in the method of the above embodiment, and will not be elaborated here.

[0132] The present application provides an interface call log analysis device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the interface call log analysis method in the first embodiment above.

[0133] Reference is made below Figure 5 , which shows a schematic structural diagram of an interface call log analysis device suitable for implementing the embodiments of the present application. The interface call log analysis device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions: tablet computers), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The interface call log analysis device shown is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present application.

[0134] As Figure 5As shown in the figure, the interface call log analysis device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the interface call log analysis device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the interface call log analysis device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an interface call log analysis device having various systems, it should be understood that it is not required to implement or have all the shown systems. Instead, more or fewer systems may be implemented or had.

[0135] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above functions defined in the methods of the embodiments disclosed in the present application are executed.

[0136] The interface call log analysis device provided by the present application adopts the interface call log analysis method in the above embodiments, and can solve the technical problem of high complexity in the deployment and configuration of log analysis tools. Compared with the prior art, the beneficial effects of the interface call log analysis device provided by the present application are the same as those of the interface call log analysis method provided by the above embodiments, and other technical features in the interface call log analysis device are the same as those disclosed in the method of the previous embodiment, and will not be elaborated here.

[0137] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0138] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0139] This application provides a storage medium, which is a computer-readable storage medium and has computer-readable program instructions (i.e., computer programs) stored thereon. The computer-readable program instructions are used to execute the interface call log analysis method in the above embodiments.

[0140] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM: Random Access Memory), read-only memory (ROM: Read Only Memory), erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or combined with an instruction execution system or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (RadioFrequency), etc., or any suitable combination of the above.

[0141] The above computer-readable storage medium can be included in the interface call log analysis device; or it can exist separately and not be assembled into the interface call log analysis device.

[0142] The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed by the interface call log analysis device, the interface call log analysis device is caused to:

[0143] The service requests for the interfaces of the microservices are intercepted by filters in the interface call log analysis toolkit, and the service information of the intercepted service requests is parsed. The service information is extracted from the service requests and is used to describe the requests themselves and their contexts, including request time, requester identity, request parameters, response time, responder identity, response content, and other relevant information. The other relevant information includes user authentication information, client IP address, and request source;

[0144] Convert the service information into log data based on a preset log format;

[0145] Obtain the demand information input from the outside world, configure the collection strategy for the log data according to the demand information, and collect the target log data in the log data according to the configured collection strategy. The collection strategy includes collection at time intervals, collection by log size, collection by request path, and collection by user identity. The collection strategy obtains the collection strategy parameters input by the user through a visual interface and generates the corresponding collection strategy;

[0146] Store the target log data in a preset database, and determine a data set composed of multiple target log data in the database, and perform anomaly detection on the data set. Among them, the database includes at least one of a relational database and a NoSQL database. The step of storing the target log data in the preset database includes: if the number of service requests within a preset time period is less than a preset threshold, save the target log data in the form of a table to the relational database; if the number of service requests within a preset time period is greater than or equal to the preset threshold, save the target log data to the NoSQL database. The step of performing anomaly detection on the data set includes: dividing the data set into different data groups according to interfaces, and taking each target log data in each data group as a data object, where each interface corresponds to a data group. After the step of taking each target log data in each data group as a data object, it includes obtaining the identity of the requester of the data object, and performing user behavior analysis according to the identity of the requester. Among them, the user behavior analysis includes judging the user group according to the identity of the requester with the most occurrences in each data group, using machine learning or data mining algorithms to analyze the user group in each data group, and identifying the behavior pattern of the user group according to the analysis result. If the data object meets the preset anomaly condition, mark the data object as an abnormal log and determine that the anomaly detection result is that there is an anomaly. Before the step of storing the target log data in the preset database, it includes: if the number of log entries is too large in a short time, temporarily store the log data with the number of log entries in a virtual space. If the log size of the log data in the virtual space exceeds the preset storage size, match the log data in the virtual space with a preset collection strategy to store the log data in the virtual space in the preset database according to the preset collection strategy;

[0147] If the anomaly detection result is that there is an anomaly, output a preset warning message.

[0148] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN: Local Area Network) or a wide area network (WAN: Wide Area Network), or it can be connected to an external computer (for example, by connecting through an Internet service provider using the Internet).

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0150] The modules described in the embodiments of this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself in some cases.

[0151] The readable storage medium provided by this application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for performing the above-mentioned interface call log analysis method, and can solve the technical problem of high complexity in the deployment and configuration of log analysis tools. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by this application are the same as those of the interface call log analysis method provided by the above embodiments, and will not be elaborated here.

[0152] The present application also provides a computer program product, including a computer program which, when executed by a processor, implements the steps of the interface call log analysis method as described above.

[0153] The computer program product provided by the present application can solve the technical problem of high complexity in the deployment and configuration of log analysis tools. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the interface call log analysis method provided in the above embodiments, and will not be elaborated herein.

[0154] The above are only partial embodiments of the present application, and thus do not limit the patent scope of the present application. Any equivalent structural transformation made by using the content of the specification and drawings of the present application under the technical concept of the present application, or any direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. An interface call log analysis method, characterized in that: Applied to a microservice that has an interface call log analysis toolkit installed, the interface call log analysis method includes: Based on the filter in the interface call log analysis toolkit, the service request of the interface requesting the microservice is intercepted, and the service information of the intercepted service request is parsed, wherein the service information is extracted from the service request and is used to describe the request itself and its context information, including request time, requester identity, request parameters, response time, responder identity, response content and other related information, and the other related information includes user authentication information, client IP address and request source; Converting the service information into log data based on a preset log format; Obtaining demand information input from the outside, configuring a collection strategy for the log data according to the demand information, and collecting target log data in the log data according to the configured collection strategy, wherein the collection strategy includes collecting by time interval, collecting by log size, collecting by request path, and collecting by user identity, and the collection strategy obtains the collection strategy parameters input by the user through a visual interface and generates a corresponding collection strategy; The target log data is stored in a preset database, and a data set composed of multiple target log data in the database is determined, and anomaly detection is performed on the data set, wherein the database includes at least one of a relational database and a Nosql database, and the step of storing the target log data in the preset database includes: if the number of service requests within a preset time period is less than a preset threshold, the target log data is saved in the form of a table to the relational database; if the number of service requests within the preset time period is greater than or equal to the preset threshold, the target log data is saved to the Nosql database; the step of performing anomaly detection on the data set includes dividing the data set into different data groups according to interfaces, and taking each target log data in each data group as a data object, wherein each interface corresponds to a data group; the step of taking each target log data in each data group as a data object The step includes obtaining the identity of the requester of the data object, performing user behavior analysis based on the identity of the requester, wherein the user behavior analysis includes determining the user group based on the identity of the requester that appears most frequently in each data group, analyzing the user group in each data group using a machine learning or data mining algorithm, and identifying the behavior pattern of the user group based on the analysis result; if the data object meets a preset abnormal condition, marking the data object as an abnormal log, and determining that the abnormal detection result is an abnormality; before the step of storing the target log data in a preset database, when the number of log entries is too large in a short period of time, temporarily storing the log data of the number of log entries in a virtual space, and if the log size of the log data in the virtual space exceeds the preset storage size, matching the log data in the virtual space with a preset collection strategy, so as to store the log data in the virtual space in a preset database according to the preset collection strategy; If the abnormality detection result shows that an abnormality exists, the preset warning information is output.

2. The interface call log analysis method according to claim 1, characterized in that: The step of collecting target log data in the log data according to the configured collection strategy includes: If the collection strategy is to collect at a time interval, determine a scheduled task corresponding to the collection strategy, and collect target log data in the log data according to the scheduled task, wherein the scheduled task includes collecting log data based on a preset time interval; If the collection strategy is to collect by log size, then detecting whether the storage data volume of the log data is greater than a preset storage data volume threshold, and if it is greater than the preset storage data volume threshold, collecting the log data as target log data; If the collection strategy is to collect by log level, the log data that meets the preset log level is collected as the target log data; If the collection strategy is to collect by request path, the log data satisfying the preset request path is collected as target log data; If the collection strategy is to collect by user identity, the log data that meets the preset user identity will be collected as target log data.

3. The interface call log analysis method according to claim 1, characterized in that: After the step of taking each target log data in each data group as a data object, the method further includes: For each data group, if the number of data objects within a preset time range is greater than a preset service request number threshold, it is determined that the data objects within the preset time range meet a preset abnormal condition; and / or, Determining the data objects corresponding to the service request in the failed state in the data group, if the number of the data objects corresponding to the service request in the failed state is greater than a preset failure number threshold, determining that the data objects corresponding to the service request in the failed state meet a preset abnormal condition; and / or, Determine the return parameters corresponding to each data object in the data group. If there is a target return parameter containing a preset specific character in each of the return parameters, determine that the data object corresponding to the target return parameter meets the preset exception condition, wherein the specific character is a pre-set character, and the target return parameter containing the specific character meets the preset exception condition.

4. An interface call log analysis device, characterized in that: The microservice is configured to install an interface call log analysis toolkit, and the interface call log analysis device includes: A log collection module intercepts the service request of the interface of the microservice based on the filter in the interface call log analysis toolkit, and parses the service information of the intercepted service request, wherein the service information is extracted from the service request and is used to describe the request itself and its context information, including request time, requester identity, request parameters, response time, responder identity, response content and other related information, wherein the other related information includes user authentication information, client IP address and request source; A log format module, which converts the service information into log data based on a preset log format; A log collection module, which obtains demand information input from the outside, configures a collection strategy for the log data according to the demand information, and collects target log data in the log data according to the configured collection strategy, wherein the collection strategy includes collecting by time interval, collecting by log size, collecting by request path, and collecting by user identity. The collection strategy obtains the collection strategy parameters input by the user through a visual interface, and generates a corresponding collection strategy. The step of storing the target log data in a preset database includes temporarily storing the log data of the log number in a virtual space when the number of log items is too large in a short period of time. If the log size of the log data in the virtual space exceeds the preset storage size, the log data in the virtual space is matched with the preset collection strategy to store the log data in the virtual space in the preset database according to the preset collection strategy. The log analysis module stores the target log data in a preset database, determines a data set composed of multiple target log data in the database, and performs anomaly detection on the data set, wherein the database includes at least one of a relational database and a Nosql database, and the step of storing the target log data in the preset database includes: if the number of service requests within a preset time period is less than a preset threshold, the target log data is saved in a table to the relational database; if the number of service requests within the preset time period is greater than or equal to the preset threshold, the target log data is saved to the Nosql database; the step of performing anomaly detection on the data set includes dividing the data set into different data groups according to the interface, and taking each target log data in each data group as a data object, wherein each interface corresponds to a data group; the step of taking each target log data in each data group as a data object After the step of obtaining the data object, the method further comprises obtaining the identity of the requester of the data object, and performing user behavior analysis based on the identity of the requester, wherein the user behavior analysis comprises determining the user group based on the identity of the requester that appears most frequently in each data group, analyzing the user group in each data group using a machine learning or data mining algorithm, and identifying the behavior pattern of the user group based on the analysis result; if the data object meets a preset abnormal condition, the data object is marked as an abnormal log, and the abnormal detection result is determined to be an abnormality; before storing the target log data in a preset database, the method comprises temporarily storing the log data of the log number in a virtual space when the number of log entries is too large in a short period of time, and if the log size of the log data in the virtual space exceeds the preset storage size, matching the log data in the virtual space with a preset collection strategy, so as to store the log data in the virtual space in a preset database according to the preset collection strategy; The log warning module outputs the preset warning information if the anomaly detection result is an anomaly.

5. An interface call log analysis device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the interface call log analysis method according to any one of claims 1 to 3.

6. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the interface call log analysis method according to any one of claims 1 to 3 are implemented.

7. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the interface call log analysis method according to any one of claims 1 to 3 are implemented.

Images