A method, device and storage medium for querying interface access log collection records

Abstract

The present application discloses a method, device and storage medium for collecting, recording and querying interface access logs, including: uniformly managing the interface information and authorizations of multiple business systems online; receiving an interface request carrying a token, parsing the token using a user information interceptor to obtain user information; using an interface access log interceptor to perform pre-processing and post-processing on the user information in sequence; after the log service receives a write request for a log object, generating a unique id for the log object and storing the log object in a queue; cyclically and batch retrieving the log objects from the queue through a separate thread and writing them into a database; receiving a query request, and performing log query after generating and reconstructing a query SQL through the log service. In this way, developers only need to write interface logic, without the need for additional coding to interface with interface access logs, automatically generate structured data, store it in the database, and directly use SQL for querying without the need for additional data processing.

Description

Technical Field

[0001] The present invention relates to the technical field of log processing, and particularly to a method, device and storage medium for collecting, recording and querying interface access logs. Background Art

[0002] Currently, Log4j, Slf4j, and Logback require developers to code and output logs. Generally, the data output to files is unformatted data, and the output logs are the running logs of the entire service, making it impossible to distinguish the operation information of specific users. It is difficult to filter the behaviors of specific users or the specific users corresponding to specific behaviors from a large number of logs.

[0003] If the behavior logs and other logs are uniformly output to a log file, the person searching and analyzing must know the log format corresponding to the behavior logs to retrieve them. Otherwise, it is very time-consuming to search for one or several lines in a log file that is often several gigabytes in size, and ordinary users cannot operate. The prior art also uses ELK to collect and store the formatted output logs of Log4j, Slf4j, and Logback, but still requires developers to code and output logs. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method, device and storage medium for collecting, recording and querying interface access logs, which can automatically generate structured data, without the need for additional coding to interface with the access logs, and directly use SQL for querying without additional data processing. The specific solutions are as follows:

[0005] A method for collecting, recording and querying interface access logs includes:

[0006] Online unified management of the interface information and authorizations of multiple business systems;

[0007] Receiving an interface request carrying a token, parsing the token using a user information interceptor, and obtaining the user information and storing it in a thread local variable;

[0008] Using an interface access log interceptor to perform pre-processing and post-processing on the user information in sequence to obtain permission information, accessing the corresponding interface to construct a log object and sending it to a log service;

[0009] After the log service receives the write request of the log object, generating a unique id for the log object and storing the log object in a queue;

[0010] Cyclically and batchly taking out the log objects from the queue through a separate thread in the log service and writing them into a database;

[0011] Receive a query request, generate and reconstruct a query SQL through the log service, and perform a log query of the database according to the query statement of the reconstructed query SQL.

[0012] Preferably, in the above interface access log collection record query method provided by the embodiments of the present invention, the online unified management of the interface information and authorization of multiple business systems includes:

[0013] Log in to the basic information management service and add an interface definition online; the interface definition includes the interface address, interface request method, microservice to which the interface belongs, the functional module or menu to which it belongs, interface code, interface name, and whether logging is required;

[0014] Assign a role to the user and assign interface permissions to the role.

[0015] Preferably, in the above interface access log collection record query method provided by the embodiments of the present invention, an interface access log interceptor is used to perform preprocessing on the user information, including:

[0016] Judge whether the microservice to which the interface belongs needs to perform interface permission checking;

[0017] If it is required, check whether all interface definitions of the current microservice are loaded into the memory of the business system microservice;

[0018] If not loaded, the business system microservice uses Feign to send a request to the basic information management service to obtain all interface definition information of the business system microservice, using the parameter microservice name; after the basic information management service receives the request, it queries all interface definition information of the business system microservice from the storage using the microservice name and returns it to the business system microservice in the form of a json array;

[0019] If loaded, obtain the interface definition corresponding to the current interface address, and the business system microservice uses Feign to send a request to the basic information management service to obtain the interface definition information that the current user has the right to access the current microservice; after the basic information management service receives the request, it queries the interface definition information that meets the conditions from the storage using the microservice name and user id and returns it to the business system microservice in the form of a json array.

[0020] Preferably, in the above interface access log collection record query method provided by the embodiments of the present invention, after returning to the business system microservice in the form of a json array, it further includes:

[0021] After the business system microservice receives the interface permission set of the user, judge whether the interface definition corresponding to the current interface address exists in the user's interface permission set;

[0022] If it exists, mark the current interface definition, obtain the interface definition code, and store the code in the attribute of the request object;

[0023] If it does not exist, directly return a prompt of unauthorized access to the system user.

[0024] Preferably, in the above-mentioned interface access log collection record query method provided by the embodiments of the present invention, an interface access log interceptor is used to perform post-processing on the user information, including:

[0025] Determine whether the current microservice needs to record interface access logs;

[0026] If it is necessary to record interface access logs, determine whether the current interface definition is marked in the pre-processing;

[0027] If it is not marked, re-determine whether it is necessary to obtain all interface definitions and obtain the current interface definition information; if it is marked, obtain the interface code from the request object, and obtain the matching interface definition from all interface definitions through the interface code;

[0028] Determine whether it is necessary to record logs in the interface definition;

[0029] If it is necessary to record logs, construct a log object and send the log object to the log service through Feign; if it is not necessary to record logs, return the interface execution result to the system user.

[0030] Preferably, in the above-mentioned interface access log collection record query method provided by the embodiments of the present invention, after writing to the database, it further includes:

[0031] Return the storage status to the business system microservice to return the interface execution result to the system user;

[0032] Judge whether the Feign fallback processing captures a request exception through the business system microservice; if a request exception is captured, print the exception information to the service log file.

[0033] Preferably, in the above-mentioned interface access log collection record query method provided by the embodiments of the present invention, the generation and reconstruction of the query SQL by the log service includes:

[0034] Generate a query SQL according to the query conditions through the log service;

[0035] Use the paging interceptor to reconstruct the query SQL and change the query statement of the query SQL into a total number query statement; or,

[0036] Reconstruct the query SQL using the total number, page number, and page size per page, and change the query statement of the query SQL into a paging query statement.

[0037] Preferably, in the above interface access log collection record query method provided by the embodiments of the present invention, the log query of the database according to the query statement of the reconstructed query SQL includes:

[0038] If there is only one truly executable query statement, it is directly sent to the database for query, sorting, and paging processing;

[0039] If there is more than one truly executable query statement, the paging information in the statement is removed, multiple SQL statements are executed in parallel to obtain multiple sets, the sets are merged in reverse order according to the table name, and subsets are obtained according to the paging information.

[0040] The embodiments of the present invention also provide an interface access log collection record query device, including a processor and a memory. Among them, when the processor executes the computer program stored in the memory, the above interface access log collection record query method provided by the embodiments of the present invention is implemented.

[0041] The embodiments of the present invention also provide a computer-readable storage medium for storing a computer program. Among them, when the computer program is executed by a processor, the above interface access log collection record query method provided by the embodiments of the present invention is implemented.

[0042] As can be seen from the above technical solutions, an interface access log collection record query method provided by the present invention includes: online unified management of interface information and authorization of multiple business systems; receiving an interface request carrying a token, parsing the token using a user information interceptor to obtain user information and storing it in a thread local variable; using an interface access log interceptor to perform pre-processing and post-processing on the user information in sequence to obtain permission information, accessing the corresponding interface to construct a log object and sending it to a log service; after the log service receives a write request for the log object, generating a unique id for the log object and storing the log object in a queue; using a separate thread in the log service to loop and batch retrieve log objects from the queue and write them to a database; receiving a query request, generating and reconstructing a query SQL through the log service, and performing a log query of the database according to the query statement of the reconstructed query SQL.

[0043] Through the above method provided by the present invention, developers only need to write interface logic, without additional coding to interface with the interface access log, and can automatically generate structured data, store it in a structured database, and directly use SQL for query without additional data processing. In addition, the present invention also provides a corresponding device and computer-readable storage medium for the interface access log collection record query method, further making the above method more practical, and the device and computer-readable storage medium have corresponding advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or in the related art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the related art. Obviously, the accompanying drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained according to the provided drawings.

[0045] Figure 1 It is a flowchart of the interface access log collection record query method provided by the embodiment of the present invention;

[0046] Figure 2 It is an architecture diagram of the interface access log collection record query provided by the embodiment of the present invention. Specific embodiments

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0048] The present invention provides an interface access log collection record query method, as Figure 1 shown, including the following steps:

[0049] S101. Online uniformly manage the interface information and authorizations of multiple business systems;

[0050] In specific implementation, step S101 for online uniformly managing the interface information and authorizations of multiple business systems may specifically include: First, add interface definitions in the basic information management service (lark-admin), specifically by the administrator logging in to the basic information management platform to add interface definitions online; the interface definitions mainly include interface addresses, interface request methods, microservices to which the interfaces belong, functional modules or menus to which they belong, interface codes, interface names, and whether log recording is required; then, assign roles to users and assign interface permissions to the roles. The request methods are mainly GET and POST, and some school firewalls prohibit the use of PUT and DELETE. For addresses that need to carry parameters in the path, {*} is used to indicate the parameter position. For example, / menu / delete / {*} represents the interface address for deleting a specified menu. When used, {*} is actually the menu id. For requests that carry parameters in the param manner, only the path needs to be configured and no parameters need to be configured. For example, for the parameter-carrying address / menu / delete?id=xxxxx, the address in the interface definition only needs to be / menu / delete.

[0051] It should be noted that lark-admin contains menu information, interface information, user information, department information, position information, role information, tenant information, etc. The interface of the present invention refers to the RESTFUL interface. RESTFUL is a design style and development method of a network application, based on HTTP, and can be defined in XML format or JSON format. RESTFUL is suitable for scenarios where mobile Internet manufacturers use it as a business interface to realize the function of third-party OTT calling mobile network resources, and the action type is to add, change, and delete the called resources.

[0052] S102, receiving an interface request carrying a token, using a user information interceptor to parse the token, obtaining user information and storing it in a thread local variable;

[0053] Specifically, the user initiates an interface request with a token; any interface that requires verification of interface permissions or requires recording of interface access logs must be logged in to obtain a token before access is allowed. After the microservice to which the interface belongs receives the request, the user information interceptor parses the token (identifies the user information), obtains the user information, and stores the user information in the thread local variable.

[0054] It should be noted that token means a token (temporary) in computer identity authentication, which is generally used for invitation and login system. The above interface request can be a web interface call interface, other microservice call interface, third-party service call interface, or HTTP tool call interface such as postman and jmeter. The above user information obtained can include user name, user account, user id, tenant id, department id, etc.

[0055] S103, using the interface access log interceptor to perform pre-processing and post-processing on the user information in turn to obtain permission information, access the corresponding interface to construct a log object and send it to the log service;

[0056] It should be noted that the interface access log interceptor is a tool for intercepting http requests for authorization authentication and logging, and is developed based on SpringMVC. The interface access log interceptor is divided into pre-processing and post-processing, with the original interface processing logic in the middle, which is equivalent to intercepting and processing the parts before and after the interface call.

[0057] S104, after the log service receives the write request for the log object, it generates a unique ID for the log object and stores the log object in a queue;

[0058] It should be noted that the step of generating a unique ID for the log object by the log service (lark-log) after receiving the log object write request is a multi-process step. The ID is unique and has its own time sequence. The log object is stored in the queue, and no value is returned to the business system microservice. The log object ID can be generated using the SnowFlake algorithm to ensure uniqueness and its own time sequence.

[0059] The queue uses java.util.concurrent.LinkedBlockingQueue.LinkedBlockingQueue, with a single maximum capacity of 10,000. The lark-log is deployed in a cluster. For example, if 3 nodes are deployed, the maximum capacity of the queue at the same time is 30,000. When the business system microservice sends a log object to the lark-log cluster, it first obtains the current lark-log cluster node information from the registration center.

[0060] S105. Use a separate thread in the log service to loop and batch retrieve log objects from the queue and write them to the database;

[0061] It should be noted that a separate thread in lark-log (i.e., the thread starts when the service starts and runs until the service process dies) loops and batch retrieves log objects from the queue and writes them to the database in a multi-threaded step. In the database, the log table is partitioned by month, and a new log table for the next month is automatically generated at a fixed time each month (such as the 1st) (i.e., a scheduled task). When the service starts, it automatically checks whether the log table for the current month exists, and if not, it automatically creates it. The log table name format is: operation_log_yyyy_MM, where yyyy_MM represents the year and month. It is determined which table to write the log object to by calculating the year and month of the request time (i.e., write to the partitioned table). The lark-log is deployed in a cluster. For example, if 3 nodes are deployed, there are 3 threads independently processing their respective 3 queues at the same time.

[0062] S106. Receive a query request, generate and reconstruct a query SQL through the log service, and perform a log query on the database according to the query statement of the reconstructed query SQL.

[0063] In the above method for querying interface access log collection records provided by the embodiments of the present invention, developers only need to write the interface logic, without additional coding to interface with the interface access log, and can automatically generate structured data, store it in a structured database, and directly use SQL for querying without additional data processing.

[0064] In specific implementation, in the above method for querying interface access log collection records provided by the embodiments of the present invention, step S103 uses an interface access log interceptor to perform preprocessing on user information, which may specifically include the following steps:

[0065] Determine whether the microservice to which the interface belongs needs to perform interface permission checking; specifically, obtain the configuration item auth.useApiAuth of the configuration information; if it is true, it means that interface permissions need to be verified; if it is false, it means that verification is not required;

[0066] If interface permission checking is required, check whether all interface definitions of the current microservice are loaded into the memory of the business system microservice;

[0067] If not loaded, the business system microservice uses Feign to send a request to the basic information management service to obtain all interface definition information of the business system microservice. The parameter microservice name comes from the configuration information. The purpose is to distribute the computing pressure of interface authentication and interface log recording to each business system, reducing the memory and CPU consumption of the gateway; after the basic information management service receives the request, it uses the microservice name to query all interface definition information of the business system microservice from the storage and returns it to the business system microservice in the form of a json array. Specifically: first query from redis, if not found in redis, then query from the database, and write the result into redis after database query; the next query can directly query from redis;

[0068] If loaded, obtain the interface definition corresponding to the current interface address, and the business system microservice uses Feign to send a request to the basic information management service to obtain the interface definition information that the current user has the right to access the current microservice; after the basic information management service receives the request, it uses the microservice name and user id to query the interface definition information that meets the conditions from the storage and returns it to the business system microservice in the form of a json array. Specifically: first query from redis, if not found in redis, then query from the database, and write the result into redis after database query. The next query can directly query from redis;

[0069] If interface permission checking is not required, call the original interface processing logic.

[0070] Obtain the interface definition corresponding to the current interface address in the above steps. Specifically: first obtain the current request address and the current request method; then traverse all interface definition information, and if it matches both the current request address and the current request method, it is the interface definition corresponding to the current interface address. The request method strings are exactly equal for matching. If the request address does not contain {*}, the strings are exactly equal for matching. If the request address contains {*}, then first replace all {*} with [a-zA-Z\\d]+, then add ^ at the prefix and $ at the suffix. Construct a regular expression, and then determine whether it matches the current request address.

[0071] It should be noted that in the high-concurrency scenario of the present invention, interface logging and authentication are completed in the business system rather than in the microservice gateway, which disperses the computing pressure to each business system, improves the gateway forwarding speed, and reduces the memory and CPU consumption of the server where the gateway is located.

[0072] Furthermore, in the specific implementation, in the above-mentioned interface access log collection record query method provided by the embodiment of the present invention, after executing the above steps and returning to the business system microservice in the form of a json array, it can also include: after the business system microservice receives the user's interface permission set, it determines whether the interface definition corresponding to the current interface address exists in the user's interface permission set, that is, identifies the user behavior, who the user is, what operation was performed at what time, whether the user has the right to operate, etc.; if it exists, then mark the current interface definition, obtain the interface definition code, and store the code in the attribute apiCode of the request object (HttpServletRequest for Java); if it does not exist, then directly return to the system user to prompt that there is no access. At this point, the pre-processing is completed, and the original interface processing logic call can be executed next.

[0073] In specific implementation, in the above-mentioned interface access log collection record query method provided by the embodiment of the present invention, step S103 uses the interface access log interceptor to post-process the user information, which may specifically include the following steps:

[0074] First, determine whether the current microservice needs to record interface access logs;

[0075] If the current microservice needs to record interface access logs (such as obtaining the configuration item auth.useApiLog of the configuration file, if it is true, it means that interface access logs need to be recorded), then determine whether the current interface definition is marked in the pre-processing;

[0076] If it is not marked, re-determine whether it is necessary to obtain all interface definitions and obtain the current interface definition information; the reasons for not marking may include that the interface permission verification is not enabled, and the current interface definition needs to be obtained later (interface permissions are not verified but interface calls are recorded); the corresponding interface definition is not matched, and the interface is not defined in lark-admin, and the interface execution result is returned to the system user later;

[0077] If marked, the interface code (the value of the attribute apiCode) is obtained from the request object (HttpServletRequest for Java), and the matching interface definition is obtained from all interface definitions through the interface code;

[0078] If the current microservice does not need to record interface access logs (for example, when obtaining the configuration item auth.useApiLog in the configuration file, if it is false, it means that interface access logs do not need to be recorded), then return the interface execution result to the system user;

[0079] After that, determine whether logs need to be recorded in the interface definition;

[0080] If logs need to be recorded (for example, if the isLog attribute in the interface definition is true, it means logs are to be recorded), then construct a log object and send the log object to the log service via Feign; if logs do not need to be recorded (for example, if the isLog attribute in the interface definition is false, it means no logs are to be recorded), then return the interface execution result to the system user.

[0081] Specifically, the log object can be constructed using the interface definition, user information, timestamp, etc. The interface log information is automatically formatted instead of outputting text. The attributes of the log object can include the affiliated menu (business module), interface name (operation), interface definition address, interface definition request method, client information (userAgent from the client), actual request address (with path parameter values), client ip (from the client), request time (long - type timestamp), requesting user id (from the token), requesting user name (from the token), and requesting user tenant (from the token).

[0082] In practical applications, after constructing the log object, it can be determined whether an exception is thrown during the execution of the original interface processing. There is a parameter exception in the post - interceptor. If an exception is thrown, this value is not null. If the parameter exception is null, then send the log object to lark - log via Feign. If the parameter exception is not null, obtain its attribute value message, assign it to the exception attribute of the log object, and send the log object to lark - log via Feign.

[0083] In specific implementation, in the above - mentioned interface access log collection, recording, and query method provided by the embodiments of the present invention, after step S105 of writing to the database, it can further include: returning the storage status to the business system microservice (only obtaining the return status without obtaining any return values) to return the interface execution result to the system user; after that, the business system microservice determines whether Feign degradation processing captures a request exception; if a request exception is captured, then print the exception information to the service log file. This achieves the purpose of exception detection and display.

[0084] The Feign fallback handling code is in the lark-auth-client library, and business system developers no longer need to write relevant code. In this invention, the main processing is the degradation of the failure count, mainly for some unstable APIs. When the number of failed calls reaches a certain threshold, it will be automatically degraded. Similarly, an asynchronous mechanism is used to detect the response situation. In addition, the service log uses logback and is integrated into the Spring Boot service through the spring-boot-starter-logging library.

[0085] It should be noted that in this invention, for the configuration of the business system microservice project, the access is simple. Business system developers only need to introduce the library and configure the interceptor, and do not need to write the generation and output of interface logs additionally.

[0086] Create a new business system microservice project, and the project must be built based on Spring Cloud, Spring Boot, and Spring MVC.

[0087] The business system microservice project introduces the lark-auth-client library.

[0088] Taking the maven management of project dependencies as an example

[0089] <dependency>

[0090] <groupid>com.mht.lark< / groupid>

[0091] <artifactid>lark-auth-client< / artifactid>

[0092] <version> 3.2< / version>

[0093] < / dependency>

[0094] Configure the interface access log interceptor in the business system microservice project.

[0095] Use the org.springframework.context.annotation.Configuration annotation in Spring to define the configuration.

[0096] Construct an instance apiAuthRestInterceptor of the com.mht.lark.security.auth.client.interceptor.ApiAuthRestInterceptor class (the interface access log interceptor is defined in the lark-auth-client library). Register the instance apiAuthRestInterceptor into the instance of the org.springframework.web.servlet.config.annotation.InterceptorRegistry class and use the addInterceptor method of InterceptorRegistry.

[0097] The interface access log interceptor should be registered in the InterceptorRegistry later than the user information interceptor, and executed later to ensure that the user information is parsed before recording the user interface call behavior.

[0098] In a specific implementation, in the above-mentioned interface access log collection record query method provided in the embodiment of the present invention, before executing S106 to receive the query request, the following steps may also be included:

[0099] The administrator uses the browser to open the log query page (menu "Operation Log"), and after the page is loaded, JS sends a request to lark-admin to obtain all current business systems;

[0100] After receiving the request, lark-admin reads all the added business system information from the database and returns it to the browser in the form of a JSON collection;

[0101] After the browser receives all the returned business system data, it uses JS to render the business system data as a select drop-down selection, waiting for the administrator to select a business system;

[0102] After the administrator selects a business system, JS monitors the change of the select value and automatically sends a request to lark-admin to obtain the microservice information under the business system, carrying the parameter business system id;

[0103] After receiving the request, lark-admin obtains all microservice information under the business system from the database according to the business system ID, and returns it to the browser in the form of a JSON collection;

[0104] After the browser receives all the microservice data under the returned business system, it uses JS to render the business system data as a select drop-down selection. Wait for the administrator to select a microservice;

[0105] After the administrator selects a microservice, JS listens to changes in the select value and then renders the date range selector and the module input box to which it belongs. The date range selector value must be filled in and cannot be left empty, because the log table stores the sub-table field as time. If the time is not passed, the memory and time costs of paging all table data will be high. The default start date and end date of the date range selector value are both today. The maximum online query time range of the menu "Operation Log" is the most recent two months, which means that data can be queried, sorted, and paginated from at most two tables at a time. If the query exceeds the range of the most recent two months, query by table name. You need to select a specific historical table in the menu "Operation Log Table" to query data, and you can only query in one table at a time. The module input box is not required.

[0106] When the administrator clicks "Query", JS sends a request to lark-log to query the paginated log, carrying parameters such as microservice name, start date, end date, user token, function module or menu (optional), page number, and page size. The start date and end date are string types in the format of yyyy-MM-dd.

[0107] Next, after receiving the request, lark-log first calculates the time range and converts the string date into timestamp milliseconds. The end date needs to be added one day.

[0108] In specific implementation, in the above-mentioned interface access log collection record query method provided by the embodiment of the present invention, step S106 generates and reconstructs the query SQL through the log service, which may specifically include:

[0109] First, the query SQL (original query statement) is generated through the log service according to the query conditions; the query conditions can include microservice name field = microservice name AND request time field > = start timestamp AND request time field < end timestamp AND tenant field = user tenant id. If the module exists, the condition is also added: AND module field LIKE '% module%'. Sorting is in reverse order by object id;

[0110] Then, the query SQL is reconstructed using the paging interceptor to change the query statement of the query SQL into a total query statement; or, the query SQL is reconstructed using the total number, page number, and size of each page to change the query statement of the query SQL into a paging query statement.

[0111] Specifically, the paging interceptor first reconstructs the SQL, transforms the original query statement into a total count query statement, and executes the total count query statement to obtain the total count that meets the conditions. Calculate the table names to be queried based on the time range in the query statement, with a maximum of two tables. Replace the logical table names with the real table names to obtain the real executable total count query statement. If there are two tables, two real executable total count query statements will be obtained. If there is only one real executable total count query statement, it will be directly sent to the database to query the total count. If there is more than one real executable total count query statement, multiple SQL statements will be executed in parallel to obtain multiple total counts. Then add them up to get the final total count. And use the total count, page number, and page size to reconstruct the SQL, transform the original query statement into a paging query statement, and execute the paging query statement. Calculate the table names to be queried based on the time range in the query statement, with a maximum of two tables. Replace the logical table names with the real table names to obtain the real executable query statement. If there are two tables, two real executable query statements will be obtained. Execute the real executable query statement. If there is only one real executable query statement, it will be directly sent to the database for querying, sorting, and paging processing. If there is more than one real executable query statement, first remove the paging information in the statements and then execute multiple SQL statements in parallel to obtain multiple sets. Merge the sets in reverse order according to the table names, and then obtain the subsets according to the paging information. Finally, the total count and the set of the current page are returned to the browser in the form of a json object.

[0112] Correspondingly, an interface access log collection record query device according to an embodiment of the present invention is also disclosed, including a processor and a memory; wherein, when the processor executes the computer program stored in the memory, the interface access log collection record query method disclosed in the foregoing embodiment is implemented.

[0113] For a more specific process of the above method, reference can be made to the corresponding content disclosed in the foregoing embodiment, and details will not be repeated here.

[0114] Further, the present invention also discloses a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, the interface access log collection record query method disclosed above is implemented.

[0115] For a more specific process of the above method, reference can be made to the corresponding content disclosed in the foregoing embodiment, and details will not be repeated here.

[0116] In this specification, the various embodiments are described in a progressive manner. The key points of each embodiment are the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the devices and storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple. For the relevant parts, reference can be made to the description of the method part.

[0117] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0118] The steps of the methods or algorithms described in combination with the embodiments disclosed herein can be directly implemented by hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0119] In summary, an interface access log collection, recording, and query method provided by an embodiment of the present invention includes: online unified management of interface information and authorization of multiple service systems; receiving an interface request carrying a token, parsing the token using a user information interceptor to obtain user information and storing it in a thread local variable; using an interface access log interceptor to perform pre-processing and post-processing on the user information in sequence to obtain permission information, accessing the corresponding interface to construct a log object and sending it to a log service; after the log service receives a write request for the log object, generating a unique id for the log object and storing the log object in a queue; using a separate thread in the log service to loop and batch retrieve log objects from the queue and write them to a database; receiving a query request, generating and reconstructing a query SQL by the log service, and performing a log query on the database according to the query statement of the reconstructed query SQL. In this way, developers only need to write interface logic and do not need to code additionally to interface with the interface access log, and can automatically generate structured data, store it in a structured database, and directly use SQL for query without additional data processing. In addition, the present invention also provides a corresponding device and computer-readable storage medium for the interface access log collection, recording, and query method, further making the above method more practical, and the device and computer-readable storage medium have corresponding advantages.

[0120] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent in such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0121] The above has introduced in detail the interface access log collection record query method, device and storage medium provided by the present invention. Specific examples are used in this text to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for querying interface access log collection records, characterized in that, Including: Online unified management of interface information and authorization for multiple business systems; Receiving interface requests carrying tokens, using a user information interceptor to parse the tokens, obtaining user information and storing it in thread local variables; Using an interface access log interceptor to perform pre-processing and post-processing on the user information in sequence to obtain permission information, accessing the corresponding interfaces to construct log objects and sending them to the log service; The interface access log interceptor is a tool for intercepting HTTP requests for permission authentication and log recording; Among them, using the interface access log interceptor to perform pre-processing on the user information includes: judging whether the microservice to which the interface belongs needs to perform interface permission checks; if so, checking whether all interface definitions of the current microservice are loaded into the memory of the business system microservice; if not loaded, using the business system microservice to send a request to the basic information management service through Feign to obtain all interface definition information of the business system microservice, using the parameter microservice name; after the basic information management service receives the request, using the microservice name to query all interface definition information of the business system microservice from the storage and returning it to the business system microservice in the form of a JSON array; if loaded, obtaining the interface definition corresponding to the current interface address, and using the business system microservice to send a request to the basic information management service through Feign to obtain the interface definition information that the current user has the right to access the current microservice; after the basic information management service receives the request, using the microservice name and user ID to query the qualified interface definition information from the storage and returning it to the business system microservice in the form of a JSON array; Using the interface access log interceptor to perform post-processing on the user information includes: judging whether the current microservice needs to record interface access logs; if it is necessary to record interface access logs, judging whether the current interface definition is marked in the pre-processing; if not marked, re-judging whether it is necessary to obtain all interface definitions and obtaining the current interface definition information; if marked, obtaining the interface code from the request object and obtaining the matching interface definition from all interface definitions through the interface code; judging whether it is necessary to record logs in the interface definition; if it is necessary to record logs, constructing a log object and sending the log object to the log service through Feign; if it is not necessary to record logs, returning the interface execution result to the system user; After the log service receives the write request of the log object, generating a unique ID for the log object and storing the log object in a queue; Using a separate thread in the log service to loop and batch retrieve the log objects from the queue and write them to the database; Receiving a query request, generating and reconstructing a query SQL through the log service, and performing log queries on the database according to the query statement of the reconstructed query SQL.

2. The method for querying interface access log collection records according to claim 1, wherein, The online unified management of interface information and authorization for multiple business systems includes: Add interface definitions to the basic information management service; the interface definitions include interface addresses, interface request methods, microservices to which the interfaces belong, functional modules or menus to which they belong, interface codes, interface names, and whether logging is required. Assign roles to users and assign interface permissions to the roles.

3. The interface access log collection record query method according to claim 2, wherein After returning to the business system microservice in the form of a json array, it also includes: After the business system microservice receives the set of interface permissions of the user, determine whether the interface definition corresponding to the current interface address exists in the set of interface permissions of the user. If it exists, mark the current interface definition, obtain the interface definition code, and store the code in the attributes of the request object. If it does not exist, directly return a prompt to the system user that they have no permission to access.

4. The interface access log collection record query method according to claim 3, wherein After writing to the database, it also includes: Return the storage status to the business system microservice to return the interface execution result to the system user. Determine whether the Feign fallback handling captures a request exception through the business system microservice; if a request exception is captured, print the exception information to the service log file.

5. The interface access log collection record query method according to claim 1, wherein The generation and reconstruction of the query SQL by the log service includes: Generate a query SQL according to the query conditions by the log service. Use a paging interceptor to reconstruct the query SQL and change the query statement of the query SQL to a total count query statement; or, Reconstruct the query SQL using the total count, page number, and page size per page, and change the query statement of the query SQL to a paging query statement.

6. The interface access log collection record query method according to claim 5, wherein The log query of the database according to the query statement of the reconstructed query SQL includes: If there is only one truly executable query statement, directly send it to the database for query, sorting, and paging processing. If there is more than one truly executable query statement, remove the paging information in the statements, execute multiple SQLs in parallel, obtain multiple sets, merge the sets in reverse order by table name, and obtain subsets according to the paging information.

7. An interface access log collection record query device, characterized in that, It includes a processor and a memory. Among them, when the processor executes the computer program stored in the memory, it implements the interface access log collection record query method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, For storing a computer program, where the computer program, when executed by a processor, implements the interface access log collection record query method according to any one of claims 1 to 6.

Images