Audit log collection method, apparatus, program product

Abstract

The application provides an audit log collection method and device, a computer program product, and an audit middleware is used to intercept an HTTP request, the audit middleware is configured in a request processing chain of a Web framework, and the audit middleware performs interception before the HTTP request reaches a business processing function and after a response is returned; it is judged whether an HTTP method of the HTTP request is a write operation method; if the HTTP method is the write operation method, audit information of the HTTP request is collected, and the audit information at least includes user identity information and request context information; after the response of the HTTP request is returned to a client, an asynchronous concurrent task is started to write the audit information into an audit log storage system in a non-blocking manner. The application skillfully uses the middleware characteristics configured by the Web framework itself, realizes "zero invasion" and "high performance" of the audit function in a light manner, and solves the pain points of code redundancy, performance loss and incomplete audit in the traditional scheme.

Description

Technical Field

[0001] This application relates to the field of security auditing technology, specifically to an audit log collection method, electronic device, and computer program product. Background Technology

[0002] To meet compliance requirements (such as the Cybersecurity Law and the graded protection system) and to enable post-incident traceability of security incidents, enterprise information systems need to audit and record user operations, especially write operations involving data creation, modification, and deletion.

[0003] In existing technologies, a common audit log collection scheme involves manually inserting audit log recording statements into the code of each business processing function. This means that after executing the core business logic, the log library's write method is called to record the operation information. However, this scheme results in a high degree of coupling between audit logic and business logic. When the system contains dozens or even hundreds of API interfaces, similar logging code must be repeatedly written in each business processing function, leading to severe code redundancy. When adding new interfaces, developers are highly likely to overlook audit code, resulting in insufficient audit coverage. Furthermore, adjusting the audit strategy requires modifying the logging statements in all business processing functions one by one, resulting in extremely high maintenance costs.

[0004] Another existing approach uses a database proxy middleware, deploying a separate proxy layer between the application and the database to intercept and parse all SQL statements, recording audit logs categorized by SQL type. However, introducing an additional database proxy layer increases system architecture complexity and deployment / maintenance costs. Network jumps in the proxy layer introduce additional request latency, and this approach only audits at the SQL statement level, failing to correlate with application-layer context information of HTTP requests—such as the operator's user identity (UserID, UserName), client IP address, User-Agent, request duration, and HTTP response status codes—information crucial for post-event traceability in security audits. Summary of the Invention

[0005] In view of this, this application provides an audit log collection method, an electronic device, and a computer program product.

[0006] According to a first aspect of the embodiments of this application, an audit log collection method is provided, including: HTTP requests are intercepted using an audit middleware configured in the request processing chain of the Web framework. The audit middleware performs the interception before the HTTP request reaches the business processing function and after the response is returned. Determine whether the HTTP method of the HTTP request is a write operation method; If the HTTP method is a write operation method, then the audit information of the HTTP request is collected, and the audit information includes at least user identity information and request context information; After the response to the HTTP request is returned to the client, an asynchronous concurrent task is started to write the audit information to the audit log storage system in a non-blocking manner.

[0007] According to a second aspect of the embodiments of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described in the first aspect.

[0008] According to a third aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0009] This application proposes technical solutions for "automatic interception using middleware built into the Web framework" and "asynchronous non-blocking writing of audit logs".

[0010] Firstly, by using audit middleware to intercept HTTP requests, the auditing function is completely separated from the specific business code and mounted as an independent component on the request processing chain. This design achieves complete decoupling between business logic and auditing logic. Developers do not need to write any auditing code in business functions. When the auditing strategy needs to be adjusted, only the middleware code needs to be modified, which greatly reduces the maintenance difficulty and error risk of the system.

[0011] Meanwhile, after the HTTP request response is returned to the client, an asynchronous thread or coroutine is started to write the audit log to the audit log storage system in a non-blocking manner. This "asynchronous write after response" strategy eliminates the impact of database I / O time on the main business process, allowing the business interface to return a response to the client immediately after processing the core logic, without waiting for the log to be written to the database. This significantly improves the throughput of the interface and effectively avoids system lag caused by log writing bottlenecks.

[0012] Furthermore, by determining whether the HTTP request method is a write operation method, the present invention only collects audit information when it is confirmed to be a write operation. This mechanism can automatically filter out a large number of read operation requests, ensuring that the audit log only contains key operations that actually change the system state. This not only avoids the waste of storage space, but also makes subsequent audit queries more focused and efficient.

[0013] Finally, the audit middleware executes before the request reaches the business processing function and after the response is returned. This full lifecycle interception mechanism ensures that the middleware can completely capture the input and output of the request, guaranteeing that the audit information covers the entire process of the operation and providing a complete and accurate chain of evidence for subsequent security tracing.

[0014] In summary, this application cleverly utilizes the middleware features configured by the Web framework itself to achieve "zero intrusion" and "high performance" of the auditing function in a lightweight manner, solving the pain points of code redundancy, performance loss and incomplete auditing in traditional solutions. Attached Figure Description

[0015] Figure 1 This is a partial flowchart illustrating an audit log collection method according to an exemplary embodiment of this application; Figure 2 This is a partial flowchart illustrating an exemplary embodiment of this application for determining read / write operations; Figure 3 This is a signaling interaction diagram illustrating an exemplary embodiment of this application for obtaining authentication information and context; Figure 4 This is a signaling interaction diagram illustrating an asynchronous writing of an audit log, as shown in an exemplary embodiment of this application. Figure 5 This is a logic block diagram of an electronic device illustrated in an exemplary embodiment of this application. Detailed Implementation

[0016] In enterprise application systems built on web frameworks, a layered architecture is typically used. Clients (such as browsers, mobile applications, or third-party services) send requests to the server via the HTTP protocol. After receiving the request, the server dispatches it to the corresponding business processing function (Handler) through the routing layer. After executing the core business logic, it returns an HTTP response. In this process, the server deploys multiple middleware to form a request processing chain. Middleware can perform preprocessing before the request reaches the business processing function and perform post-processing operations after the response is returned. This middleware mechanism is a standard built-in capability of mainstream web frameworks such as Gin, Spring Boot, and Django.

[0017] Although the middleware mechanisms built into web frameworks theoretically possess request interception capabilities, those skilled in the art have long refrained from directly implementing audit log collection using middleware, primarily due to several technical obstacles. First, middleware is designed to execute general processes orthogonal to business logic (such as request log printing, cross-domain handling, and rate limiting). These processes typically only need to be executed at a single point before or after the request. However, audit log collection requires performing a dual operation on the same request: recording the start time before the request and recording the response status and calculation time after the request. This necessitates the middleware's ability to execute collection logic at both the beginning and end of the request processing chain, placing specific demands on the middleware's execution sequence design. Second, audit logs need to be associated with authenticated user identity information. However, authentication logic is usually handled by independent authentication middleware. There is no standardized built-in solution in middleware mechanisms for efficiently and securely transferring user identity data between different middleware without increasing coupling. Developers need to design their own context passing protocols, which increases the complexity of implementation. Third, writing audit logs involves database I / O operations. If the write is performed synchronously in the middleware, it will directly block the request-response process, leading to a significant increase in interface latency. If asynchronous writing is used, it is necessary to handle the complex synchronization problem between the coroutine lifecycle and the request context lifecycle—the request may have ended when the asynchronous coroutine starts, and the request object in the context may have been recycled. How to ensure that the audit data remains valid after the context is destroyed is a technical challenge that requires special design. These multiple obstacles make those skilled in the art more inclined to use relatively direct but flawed solutions such as business code instrumentation or independent proxy layers when facing audit requirements, rather than directly utilizing middleware mechanisms.

[0018] Therefore, this application designs a middleware interception and asynchronous writing scheme, enabling audit log collection capabilities to be non-intrusively integrated into the request processing chain. Simultaneously, it achieves accurate identification of write operations, automatic collection of context information throughout the request lifecycle, and non-blocking writing decoupled from the business response process, effectively applying it to security auditing scenarios of enterprise-level application systems built on Web frameworks. The following, in conjunction with the appendix... Figure 1 The specific implementation methods of this application will be described in detail.

[0019] S101, intercepting HTTP requests using audit middleware. This audit middleware is configured in the request processing chain of the Web framework and executes before the HTTP request reaches the business processing function and after the response is returned. S102, Determine whether the HTTP method of the HTTP request is a write operation method; S103, If the HTTP method is a write operation method, then collect the audit information of the HTTP request. The audit information includes at least user identity information and request context information. S104. After the response to the HTTP request is returned to the client, an asynchronous concurrent task is started (S105) to write the audit information to the audit log storage system in a non-blocking manner (S106).

[0020] The audit middleware in S101 is typically a standard component included with modern web development frameworks (such as Spring Boot, Django, or Express). Its key feature is its "onion model" or pipelined processing mechanism, enabling preprocessing before requests enter core business logic and post-processing after response generation. This solution cleverly leverages this existing mature mechanism to address specific audit log collection issues. Traditional auditing often intrudes into business code, resulting in high coupling between business logic and logging. This application, however, achieves complete decoupling of audit logic from business logic through an audit middleware configured in the request processing chain. Regardless of the number of backend business interfaces, auditing can be seamlessly integrated by uniformly intercepting at the middleware level.

[0021] In practice, when a client initiates an HTTP request, the web server first passes the request to the audit middleware. The audit middleware captures the request before the business controller, recording basic information such as the request's start time and path, and then allows the request to enter the subsequent business processing function. After the business processing is completed and a response is generated, the control flow returns to the audit middleware. At this point, the audit middleware can obtain the final response status code and time elapsed, thus completing a full request lifecycle interception. In one embodiment, the audit middleware only operates on requests on registered routes. For 404 requests that do not match any route (e.g., malicious scanning or incorrect request paths), the Gin framework's NoRoute handler handles them independently; these requests do not enter the middleware chain and therefore do not trigger the audit middleware. If the system needs to record such requests without matching routes, logging can be implemented separately in the NoRoute handler. This scenario belongs to the security protection layer of auditing, which is a different audit dimension from the middleware write operation auditing described in this invention. The two can be deployed independently and do not affect each other.

[0022] Before performing any specific data collection actions, the S102 judgment step is required. This is because in a web system, the vast majority of requests are likely read-only query operations (such as GET requests). If deep audit information collection and storage were performed on all requests, it would generate a massive amount of invalid logs, greatly wasting storage space and increasing the system's I / O burden. Therefore, distinguishing between read and write operations is a crucial preliminary step for improving audit efficiency.

[0023] Specifically, this embodiment determines the operation by parsing the Method field in the HTTP header. Methods such as POST, PUT, DELETE, and PATCH are typically defined as write operations because they imply the creation, modification, or deletion of server resources. For example, when the Method field value is detected as "POST", it is determined to be a write operation. The following section will further elaborate on this. Figure 2 Detailed description.

[0024] As another implementation, the determination can also be made by parsing the URL path characteristics of the request. For example, if the URL contains keywords such as " / update / " or " / delete / ", or if the query parameters contain "action=write", then it is determined to be a write operation.

[0025] In contrast, the HTTP-based judgment method adopted in this embodiment has more significant advantages. The HTTP method is a standard definition at the protocol level, possessing universality and standardization, and is independent of specific URL naming conventions. However, judgment based on URL paths is easily influenced by developers' naming habits (for example, some developers might use " / modify / " instead of " / update / "), leading to missed or incorrect judgments and higher maintenance costs. Therefore, the HTTP-based judgment method is more robust and reliable.

[0026] As an example, if the judgment result is negative, that is, the HTTP request is not a write operation method (for example, a normal GET query request), then as an additional step, the audit middleware directly allows the request without performing subsequent audit information collection and storage operations, or only records a very small amount of access count information, thereby achieving rapid filtering of non-critical traffic.

[0027] In S103, once a write operation is confirmed, the system enters the substantive information collection phase. Here, user identity information refers to the identifier of the entity initiating the operation, such as the UserID parsed from the Token extracted from the HTTP Header, or the username in the Session. Request context information refers to a snapshot of the environment at the time the operation occurred. Specifically, this includes, but is not limited to: the target URL of the request, the payload parameters (such as the JSON body), the client IP address, the User-Agent, and the current timestamp. This information together constructs an audit evidence chain, ensuring that the operation scenario can be reconstructed for future tracing. Since this is at the middleware layer, all this data can be directly obtained from the Request object without requiring additional parameters from the business logic.

[0028] Step S104 ensures high performance in the audit process. After collecting audit information, the main thread's primary task is to return the business response to the user as quickly as possible, avoiding any delays due to log writing. Therefore, this step employs an asynchronous concurrency mechanism. Specifically, the system starts an independent asynchronous thread or coroutine. This asynchronous task is decoupled from the main request's processing chain, encapsulating the collected audit information into a message object and delivering it to an in-memory queue or directly sending it to a message broker (such as Kafka or RabbitMQ). By separating time-consuming disk I / O or network I / O operations (log writing) from the main process, the main thread can immediately release resources to handle the next HTTP request. Even if the audit log storage system is temporarily slow or unavailable, it will not block normal user access, thus ensuring high availability and low latency for core business operations.

[0029] like Figure 2 The image shows a specific example of determining whether an HTTP request is a read or write operation.

[0030] In S201, retrieve the HTTP method field of the HTTP request; S202, determine if the HTTP method is any of POST, PUT, or DELETE; if yes, execute S203 to determine it is a write operation method and proceed to S204 audit log collection process; if no, further execute S205 to determine if the HTTP method is GET; if yes, execute S206 to determine it is a read operation method and execute S207 to allow the HTTP request without triggering audit log recording; if no, execute S208 to process according to the preset strategy.

[0031] In real-world high-concurrency web systems, simply intercepting requests is insufficient. Without fine-grained differentiation of request types, the system may face performance bottlenecks. This is because in common business systems, query-type read operations often account for the vast majority of traffic, while data change operations that truly need to be audited are relatively few. Deeply collecting information from all traffic would lead to significant resource waste. Therefore, this embodiment achieves on-demand allocation of audit resources through specific protocol field analysis.

[0032] The audit information of the HTTP request can be collected through the following process: before the audit middleware calls the business processing function, record the start time of the HTTP request; after the audit middleware calls the business processing function, calculate the processing time of the HTTP request; collect at least one of the following information of the HTTP request: client IP address, User-Agent request header, requested resource URL path, and HTTP response status code.

[0033] In practical web architectures, auditing modules are typically located outside of business logic. If the auditing middleware directly parses the token or queries the database to obtain the username, it not only increases system latency but also leads to strong coupling between the auditing code and authentication logic (for example, if JWT authentication is changed to OAuth2 in the future, the auditing code will also need to be rewritten). Therefore, this application adopts a decoupled transmission strategy for user information, that is, using the request context of the web framework as the data carrier to achieve complete separation of authentication and auditing. In addition, in order to obtain complete audit elements including time consumption and status codes, this application utilizes the onion model characteristics of the middleware, combined with the above decoupling scheme, to build a standardized context collection process.

[0034] The following is in conjunction with the appendix Figure 3 The specific implementation process is explained in detail: As an example, user identity information can be obtained in the following ways: Configure an authentication middleware in the request processing chain. The authentication middleware extracts and parses the user identity credentials from the HTTP request, and injects the parsed user identity information into the request context of the Web framework so that the audit middleware can read the user identity information from the request context.

[0035] like Figure 3 As shown, to achieve the above decoupled design, the system configures an Auth authentication middleware at the front end of the request processing chain. When the client sends an HTTP request carrying a JWT token (S301), the Auth authentication middleware first extracts and parses the JWT token from the Authorization header (S302). Subsequently, in S303, the Auth authentication middleware further parses the Claims portion of the token, extracting key identity identifiers such as userID, username, and role.

[0036] In step S304, the Auth authentication middleware does not directly handle business logic. Instead, it calls the framework's provided setting methods (such as c.Set) to inject this information into the Gin request context as key-value pairs (with keys "userId", "userName", and "role"). At this point, step S305 demonstrates the context's role as the "sole bridge between the authentication middleware and the audit middleware." This design allows the downstream AuditLog middleware to directly read the user's identity information from the context in step S307 using methods like c.Get("userId") (S308), achieving zero intrusion of authentication logic changes into the audit module.

[0037] As an example, the authentication middleware injects the parsed user identity information into the request context of the web framework, specifically including: The authentication middleware associates and stores at least one of the user ID, username, and user role with the current request using the context setting method provided by the Web framework; the auditing middleware reads the user identity information associated with the current request using the context retrieval method provided by the Web framework.

[0038] Combination Figure 3 In the S304 step of the process, the Auth authentication middleware specifically executes the c.Set("userId",userID), c.Set("userName", userName), and c.Set("role", role) operations. This leverages the concurrent secure Map feature of the Gin framework's Context to bind and store the three core pieces of information—user ID, username, and user role—with the current HTTP request lifecycle.

[0039] Accordingly, in step S307, the AuditLog middleware reads the aforementioned information associated with the current request by calling c.Get("userId"), c.Get("userName"), and c.Get("role"). This key-value-based access method ensures that user identity data from different requests will not interfere with each other in high-concurrency scenarios, guaranteeing the accurate attribution of operator information in the audit log.

[0040] After resolving the issue of user identity information source, the execution characteristics of the "onion model" are used to address the problem of how to collect complete context, including time and environmental dimensions. For example... Figure 3 As shown, the AuditLog audit middleware is located after the Auth middleware and before the business Handler. Its execution logic is divided into two stages, "before request" and "after response", by the c.Next() call.

[0041] Specifically, before c.Next() is called in step S310 and passed to the business Handler (i.e., the push-in phase of the onion model), the audit middleware immediately records the current system time as the start time of the HTTP request and collects basic environmental information such as the client IP address, User-Agent request header, and requested resource URL path from the Request object. At this time, since the context has been passed in step S306, the audit middleware can also synchronously obtain the aforementioned user identity information.

[0042] It's important to note that in real-world production environments, web applications are typically deployed behind reverse proxies (such as Nginx or Apache). In this case, the audit middleware obtains the proxy server's IP address by default using `c.ClientIP()`, not the real client's IP address. To ensure that the audit logs record the true client origin information, enabling accurate identification of the request initiator during post-event tracing, a trusted proxy list needs to be configured during Gin framework engine initialization. For example, this can be done by calling `r.SetTrustedProxies([]string{"127.0.0.1", "::1"})` to specify the IP range of trusted proxy servers. Simultaneously, the reverse proxy needs to be configured to forward requests to the real IP address. For example, in Nginx, this can be done by configuring `proxy_set_headerX-Real-IP $remote_addr;` and `proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;`. With the coordination of these two configurations, the audit middleware can correctly obtain the client's real IP address.

[0043] After the business handler in step S311 finishes processing and returns an HTTP response (i.e., the pop-out phase of the onion model), the control flow returns to the audit middleware. At this point, the audit middleware can obtain the final HTTP response status code and accurately calculate the processing time of the HTTP request by subtracting the previously recorded "start time" from the current time. Finally, in step S309, the audit middleware aggregates user identity information (from the context), basic environment information (from the Request), response status code, and processing time (from the latter half of the onion model) to construct a complete audit log containing seven dimensions.

[0044] In real-world production environments, the persistence of audit logs (such as writing to MySQL, Elasticsearch, etc.) typically involves network I / O and disk I / O, with uncertain latency and susceptibility to database load fluctuations. If synchronous writing is used, database jitter or response delays will cause API timeouts, severely impacting user experience. Therefore, as an example, a completely new coroutine (Goroutine in Go) can be started—hereafter referred to simply as "coroutine" or "Goroutine"—as an asynchronous concurrency mechanism. Figure 4The illustrated process achieves physical isolation between business responses and the audit log database, thereby maximizing system throughput while ensuring data integrity. This embodiment uses Go's Goroutine as an example. In other programming languages, equivalent lightweight concurrency mechanisms can be used, such as Kotlin's Coroutine, Python's asyncio Task, and Rust's Tokio Task, whose technical essence is the same as the Goroutine mechanism in this embodiment.

[0045] Combined with appendix Figure 4 The specific implementation process of this asynchronous write mechanism is as follows: When the client initiates an HTTP request (S401), the main request coroutine receives and processes the business logic. At this time, the audit middleware and authentication middleware have completed the collection and construction of audit data. Before the main request coroutine ends, the system first executes step S402, encapsulating the collected user ID, operation type, resource information, IP address, time consumption, and other multi-dimensional data into a complete audit log data structure (Struct). This step is completed in memory, so it is short. Then, step S403 is entered, and the main request coroutine returns an HTTP response to the client. At this time, the user has received the response, but the database has not yet been written. This means that regardless of whether the subsequent log writing is successful or how long it takes, it will not affect the user's perception, ensuring the high availability of the interface.

[0046] After the response is sent, the system initiates an independent concurrent task via step S404. The main coroutine calls the `go` keyword or a similar asynchronous execution mechanism in the Go language to start a brand new Goroutine. This Goroutine has its own stack space and execution flow, takes over the audit log data structure, and performs the specific database write operation in step S405. Subsequently, in step S406, the asynchronous task sends an INSERT request to the audit log storage database. To further improve the reliability of audit data and cope with abnormal scenarios such as temporary database unavailability or connection timeouts, this embodiment configures a three-level fault tolerance mechanism in the asynchronous write coroutine. The first level is exponential backoff retries: after the first write failure, it automatically retryes up to 3 times, with retry intervals of 100ms, 200ms, and 400ms, to cope with temporary failures such as database connection pool exhaustion, lock waiting, or network jitter. The second level is local log fallback: If all retries still fail, the audit logs are appended to a local file (e.g., audit_fallback.log) in JSON format, ensuring that audit data is not lost due to database unavailability. Subsequent maintenance personnel can use offline scripts to add the audit records from the local file to the database. The third level is alarm notification: When the number of consecutive write failures exceeds a preset threshold (e.g., 5 times / minute), the administrator is notified through system alarm channels (e.g., Webhook callbacks, emails, SMS, etc.) to promptly intervene and handle database failures. This three-level fault tolerance mechanism ensures that even in the extreme case of complete database unavailability, audit data remains fully recoverable, and the HTTP responses of the main business process are completely unaffected by audit write failures.

[0047] It's worth noting that step S407 shows that the asynchronous Goroutine's task lifecycle ends immediately after the write operation is complete, while the main request has already finished. This design demonstrates that the lifecycle of independent concurrent tasks is independent of the HTTP request context lifecycle, effectively avoiding resource release issues caused by the main request context closing, and also preventing long-running log writing tasks from consuming valuable Web service thread resources. Through this decoupled design, the system not only achieves non-blocking writes but also reserves independent operational space for potential retry mechanisms or degradation strategies.

[0048] As an example, this application may further include a step of authentication failure auditing. This step involves starting an independent concurrent task when the authentication middleware fails to authenticate an HTTP request, asynchronously recording an authentication failure audit log; in the authentication failure audit log, the operation type field is marked as authentication failure, and the requested resource URL path is recorded.

[0049] In real-world production environments, authentication failures are often a precursor to malicious attacks (such as brute-force attacks) or system misconfigurations. If requests are rejected outright due to authentication failures, skipping the recording in audit logs, blind spots in security monitoring will occur, making it impossible for operations and maintenance personnel to trace the source of attacks or troubleshoot problems. Therefore, this implementation employs an asynchronous concurrency mechanism based on Goroutines to achieve real-time capture and asynchronous database storage of authentication failure events, thereby maximizing security observability while ensuring interface performance.

[0050] As a concrete example, when a client initiates an HTTP request, the main request goroutine receives and processes the business logic. At this point, the authentication middleware and audit middleware have completed authentication verification. If the authentication middleware determines that the token is invalid, expired, or has a signature error, the system will immediately construct a special audit log data structure. This log differs from the log of a normal request; its operation type field is explicitly marked as "authentication failed," and it fully records the requested resource URL path, client IP address, User-Agent, and the specific error reason (such as "Token expired"), providing data support for subsequent security analysis. Subsequently, the main request goroutine immediately returns a 401 Unauthorized response to the client. After the response is sent, the system starts an independent concurrent task. The main goroutine calls the `go` keyword in Go or a similar asynchronous execution mechanism to start a brand new Goroutine. This Goroutine has its own stack space and execution flow; it takes over the audit log data structure for authentication failure and performs the specific database write operation. After the write is completed, the asynchronous Goroutine's task lifecycle ends, while the main request has already finished. This design effectively avoids resource release issues caused by closing the main request context, and also prevents long-running log writing tasks from consuming valuable web service thread resources. Through this decoupled design, the system not only achieves non-blocking writing, but also reserves independent operating space for possible retry mechanisms or degradation strategies.

[0051] As an example, this application may further include the following steps: in the business processing function, calling the audit log recording interface to supplement the recording of audit information of the resource dimension entered by the user after auditing; the audit information of the resource dimension includes at least one of resource type, resource ID and resource title; the audit information collected by the audit middleware and the audit information entered by the user after auditing are written into the same audit log table.

[0052] Specifically, this embodiment adopts a two-layer audit architecture in actual deployment. The audit middleware mentioned above is located in the first layer, and the second layer is the business layer manual audit layer. During the execution of key business logic (such as creating, deleting, moving, or sharing documents) by the business processing function (i.e., the business Handler), the business Handler needs to actively call the preset audit log recording interface (for example, the business Handler actively calls the CreateOperationLog function). By calling this interface, the system can supplement the recording of audit information at the resource dimension input by the user after auditing. The supplemented audit information at the resource dimension specifically includes at least one of the following: resource type (ResourceType), resource ID (ResourceID), and resource title (ResourceTitle). In addition, in specific implementations, this interface can also selectively supplement the operation details (Detail) field to record more granular business context.

[0053] The audit information collected in this embodiment can be summarized into seven categories, covering all elements required for audit traceability: User category (including UserID and UserName, identifying the user's identity), Operation category (including Action, recording the type of operation performed), Resource category (including Resource, automatically collected by the middleware using the URL path; and ResourceType, ResourceID, and ResourceTitle, supplemented by the business layer), Client category (including IP and UserAgent, recording the device and network information from which the request originated), Response category (including Status, recording the operation result), Time-consuming category (including Duration, recording the request processing time), and Time category (including CreatedAt, recording the time the operation occurred). The audit middleware layer covers six complete dimensions: User category, Operation category, Resource category (Resource only), Client category, Response category, Time-consuming category, and Time category, as well as partial information from the Resource category. The business layer manually supplements the ResourceType, ResourceID, ResourceTitle, and Detail fields of the Resource category with the audit middleware layer. Together, these two layers constitute a complete seven-dimensional audit traceability system.

[0054] It is important to emphasize that the resource-level audit information supplemented and recorded by the business layer through the aforementioned interfaces is not written to a separate log table. Instead, it is written together with the HTTP request-level audit information automatically collected by the AuditLog audit middleware into the same audit log table. In other words, the middleware's automatic audit layer populates the URL path portions of the user, operation, client, response, time-consuming, and resource classes; the business layer's manual audit layer populates the ResourceType, ResourceID, ResourceTitle, and Detail fields of the resource class. These fields are complementary, and the records coexist in a single database table, jointly ensuring audit traceability.

[0055] Taking a specific document management system implementation as an example, when a user creates a new document via a POST request, the AuditLog middleware automatically collects HTTP-level audit information (including user ID, username, operation type, resource URL path, IP, User-Agent, status code, and time consumption) as the first record and writes it to the audit log table. Subsequently, in the business Handler, the developer actively calls the CreateOperationLog interface, passing in the resource type "document", the resource ID as the ID of the newly created document (e.g., 256), and the resource title "Project Solution v2", as the second record and writes it to the same audit log table. Thus, through two complementary records in the same table, both macro-level behavioral analysis based on the HTTP dimension and precise traceability query based on the resource dimension can be supported simultaneously.

[0056] It can be seen that the audit middleware layer and the business manual audit layer are not substitutes, but rather complementary layers. The middleware layer ensures comprehensive write operation coverage and non-intrusive automatic collection of six complete dimensions. However, due to its location, it cannot perceive the resource semantics at the business level (e.g., "what is the title of the document the user is deleting"). Therefore, the ResourceType, ResourceID, ResourceTitle, and Detail fields are left blank in the middleware layer. The business manual audit layer manually supplements these resource dimension information at key operation points, making up for the middleware layer's shortcomings at the semantic level.

[0057] In one implementation, this application may also include an audit log query step to meet post-event traceability and security auditing needs. The specific process includes: receiving an audit log query request; if the query request is a multi-dimensional combined filtering query, retrieving matching audit log records from the audit log storage system based on at least one of the filtering conditions: user identifier, operation type, and time range; if the query request is a resource-based traceability query, retrieving matching audit log records from the audit log storage system based on the resource type and resource ID.

[0058] Specifically, the system first receives audit log query requests initiated by users through the front-end management interface or API interface. After receiving the query request, the system enters two complementary query modes depending on the query type.

[0059] The first type is a multi-dimensional combined filtering query mode. When the query request is a multi-dimensional combined filtering query, the system supports free combination of at least one of the following filtering conditions for retrieval: exact matching by user identifier (userID), LIKE fuzzy matching by operation type (Action field) (e.g., searching for "POST / api / v1 / documents"), and limiting the creation time interval by time range (startTime and endTime). Simultaneously, the query interface supports pagination parameters (page number and pageSize, number of records per page) and defaults to sorting audit records in descending order of creation time. Based on the above combination of filtering conditions, the system retrieves matching audit log records from the audit log storage system (i.e., the same operation_logs audit log table mentioned above). This multi-dimensional combined filtering query can simultaneously retrieve all audit records written by the middleware automatic audit layer and the business layer manual audit layer, suitable for macro-level behavioral analysis and anomaly detection scenarios such as "viewing all operations of a user within a certain time period".

[0060] The second type is the resource-based precise traceability query mode. When the query request is a resource-based traceability query, the system performs a precise match query based on the resource type (ResourceType) and resource ID (ResourceID), retrieving all historical operation records for that specific resource from the audit log storage system (i.e., the same operation_logs table). It should be noted that since the ResourceType and ResourceID fields are empty values ​​when the middleware layer performs automatic auditing, this resource-based precise traceability query mainly hits records manually added by the business layer through calling the audit log recording interface (e.g., CreateOperationLog).

[0061] The query results can be displayed to the administrator in a table format through the front-end management interface. The columns displayed include information such as time, user, operation type, resource path, IP address, time consumed, and status code. It also supports filtering and browsing by user and operation type.

[0062] This application can be implemented by executing several computer program code flows using an electronic device. The electronic device loads the computer program into non-volatile memory and uses its processor to read these computer program instructions into memory for execution. One hardware structure diagram of the electronic device, besides... Figure 5In addition to the processor, memory, network interface, and non-volatile memory shown, electronic devices may also include other hardware depending on their actual functions, which will not be elaborated further.

[0063] Furthermore, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above embodiments. This computer program product can be distributed and deployed in the form of a software installation package, firmware, microcode, etc.

[0064] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the above embodiments. The storage medium includes, but is not limited to, various non-volatile computer-readable storage media such as ROM, RAM, hard disk, solid-state drive (SSD), optical disc (CD / DVD), and USB flash drive.

Claims

1. A method for collecting audit logs, characterized in that, include: HTTP requests are intercepted using an audit middleware configured in the request processing chain of the Web framework. The audit middleware performs the interception before the HTTP request reaches the business processing function and after the response is returned. Determine whether the HTTP method of the HTTP request is a write operation method; If the HTTP method is a write operation method, then the audit information of the HTTP request is collected, and the audit information includes at least user identity information and request context information; After the response to the HTTP request is returned to the client, an asynchronous concurrent task is started to write the audit information to the audit log storage system in a non-blocking manner.

2. The method according to claim 1, characterized in that, The audit information collected for the HTTP requests includes: Before the audit middleware calls the business processing function, record the start time of the HTTP request; After the audit middleware calls the business processing function, the processing time of the HTTP request is calculated; Collect at least one of the following information from the HTTP request: client IP address, User-Agent request header, requested resource URL path, and HTTP response status code.

3. The method according to claim 1, characterized in that, The user identity information is obtained through the following methods: An authentication middleware is configured in the request processing chain. The authentication middleware extracts and parses the user identity credentials from the HTTP request and injects the parsed user identity information into the request context of the Web framework, so that the audit middleware can read the user identity information from the request context.

4. The method according to claim 1, characterized in that, The step of initiating an asynchronous concurrent task to write the audit information to the audit log storage system in a non-blocking manner includes: Construct an audit log data structure that includes the audit information; After the response to the HTTP request has been sent to the client, an independent concurrent task is started through the asynchronous execution mechanism provided by the Web framework or programming language to perform the operation of writing the audit log data structure into the database; The lifecycle of the independent concurrent task is independent of the context lifecycle of the HTTP request.

5. The method according to claim 3, characterized in that, The method also includes an authentication failure auditing step: When the authentication middleware fails to authenticate the HTTP request, an independent concurrent task is started to asynchronously record an audit log of the authentication failure. In the audit log of the authentication failure, the operation type field is marked as authentication failure, and the requested resource URL path is recorded.

6. The method according to claim 1, characterized in that, The method further includes: In the business processing function, the audit log recording interface is called to supplement and record the audit information of the resource dimension entered by the user after auditing; the audit information of the resource dimension includes at least one of the following: resource type, resource ID, and resource title; The audit information collected by the audit middleware and the audit information entered by the user after auditing are written into the same audit log table.

7. The method according to claim 6, characterized in that, It also includes the audit log query step: Receive audit log query requests; If the query request is a multi-dimensional combined filtering query, then the matching audit log records are retrieved from the audit log storage system based on at least one of the filtering conditions: user identifier, operation type, and time range. If the query request is a resource-based traceability query, then the matching audit log record is retrieved from the audit log storage system based on the resource type and resource ID.

8. The method according to any one of claims 1 to 7, characterized in that, The web framework is the Gin framework, and the asynchronous concurrent execution task is a Goroutine.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

Images