Audit method, system and equipment based on Gin framework cloud platform
By introducing an audit middleware based on the Gin framework into the cloud platform, the issues of uniformity and high performance in cloud platform security auditing have been resolved, enabling flexible security control and efficient audit log storage, thereby improving the security and stability of the system.
Patent Information
- Application Number
- CN202411241559.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-11-12
AI Technical Summary
The lack of a unified high-performance security audit middleware in existing cloud platforms leads to inconsistent audit function technology stacks and poor performance, increasing the difficulty of development and operation and maintenance. Furthermore, the inability to unify storage components makes it difficult to adapt to constantly changing security standards and regulations.
An audit middleware is built based on the Gin framework. By generating corresponding types of audit middleware, it intercepts service requests, performs audits on network settings, identity authentication, access permissions, and log recording, and stores the audit logs in a distributed database, supporting high concurrency and flexible security control.
It enables unified security auditing on the cloud platform, improves the system's high-concurrency performance and security, reduces development and operation and maintenance costs, and supports dynamic adjustment of auditing strategies to adapt to ever-changing security needs.
Smart Images

Figure CN118944968A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of software development technology, and in particular to an audit method, system and device based on the Gin framework cloud platform. Background Art
[0002] Cloud computing platform is a new service platform based on the Internet (i.e. cloud platform). It uses virtualization technology to encapsulate computing resources, storage resources and applications into an independent virtual environment, and provides users with various remote services. It is widely used in big data processing, artificial intelligence, enterprise applications, the Internet of Things and game platforms.
[0003] In related technologies, most platforms customize and develop security audit business functions in their own projects, and there is no unified and independent cloud platform high-performance security audit middleware. Although each platform can achieve its goal by developing its own security audit process, which is also meaningful, it also faces some challenges and problems:
[0004] First, the complexity of the cloud platform makes the architectural design of security management and audit functions more difficult. It is necessary to consider how the technical architecture of the security management and audit functions can be better compatible and adapted with the cloud platform's own technical architecture, which increases the cost of technical architecture design.
[0005] Second, the cloud platform security audit process is relatively standardized and unified. If each platform is repeatedly customized and developed, it will inevitably require enterprises to invest more time and manpower costs.
[0006] Third, the security standards and regulations of cloud platforms are constantly changing and updating. Enterprises need to follow up and adjust security strategies in a timely manner, and the cost of updates and maintenance increases.
[0007] Fourth, the development languages of various platforms are not unified, and there are some languages with poor concurrency performance, such as Java and PHP.
[0008] Fifth, it is impossible to unify storage components in their respective custom developments.
[0009] In summary, the current cloud platform auditing method lacks a unified audit middleware, resulting in an inconsistent auditing technology stack, poor performance, redundant designs, and repeated development, which increases development costs and operation and maintenance difficulties. Summary of the invention
[0010] In view of the shortcomings of the prior art mentioned above, the present application discloses an audit method, system and device based on the Gin framework cloud platform to solve at least one of the above-mentioned technical problems.
[0011] The first aspect of the present application discloses an audit method based on a Gin framework cloud platform, comprising: in response to a service request for accessing the cloud platform, generating an audit middleware of a corresponding type according to the type of the service request, and storing the audit middleware in a gateway service, wherein the gateway service is constructed based on the Gin framework; calling the audit middleware to intercept the service request, auditing the service request according to preset audit rules, determining and storing the audit log, wherein the preset audit rules include at least one of network settings, identity authentication, access rights and log records.
[0012] In certain embodiments of the first aspect of the present application, after determining the audit log and storing it, it also includes: performing data analysis on the audit log to generate an analysis report, and adjusting the audit policy of the audit middleware based on the analysis report, wherein the analysis report includes the type of abnormal data, occurrence time, impact range, abnormal cause and remedial measures; if the requester sending the service request in the analysis report is a user in a preset list, determine the level of the audit policy according to the identity information of the user, adjust the audit policy according to the corresponding level and respond, and the identity information at least includes the user's position information; if the request time of any of the service requests involved in the analysis report is within a preset time period, determine the different levels of the audit policy according to the time interval difference within the preset time period, adjust the audit policy according to the corresponding level and respond.
[0013] In certain embodiments of the first aspect of the present application, an audit middleware of a corresponding type is generated according to the type of the service request, and the audit middleware is stored in a gateway service, including: building multiple middlewares based on a high-concurrency Gin framework, and sequentially organizing the multiple middlewares into a middleware chain; determining the type of the service request, and determining the location information of a matching middleware in the middleware chain according to the type of the service request, wherein the service request is an Http request; configuring the processing audit logic of the middleware corresponding to the location information through a preset hook function, determining the configured middleware as an audit middleware, and caching it to the gateway service of the local gateway.
[0014] In certain embodiments of the first aspect of the present application, the audit middleware is called to intercept the service request, the service request is audited according to preset audit rules, and the audit log is determined and stored, including: calling the audit middleware to intercept the service request, collecting the user ID, operation type, timestamp, routing information, request message and response message of the intercepted service request to form a temporary file, wherein the collection is performed through a Fluentd streaming data collector; the network settings, identity authentication, access rights and log records in the temporary file are reviewed and verified one by one according to the preset audit rules to generate an audit log; the audit log is processed in blocks to generate audit log blocks, and the audit log blocks are written to a back-end database, which is a distributed storage.
[0015] In certain embodiments of the first aspect of the present application, after determining the audit log and storing it, it also includes: using a data analysis engine to analyze the query request, and splitting the query request into multiple query sub-requests based on the analysis results; using distributed search to process multiple query sub-requests in parallel, merging the audit logs corresponding to each query sub-request according to a preset log template, and outputting log feedback results.
[0016] In certain embodiments of the first aspect of the present application, the gateway service is constructed based on the Gin framework, including: obtaining a package management tool based on a development environment of the go language, and using the package management tool to install the Gin framework; generating an executable file with a Gin application by compiling go code, creating a Docker container, and deploying the executable file in the Docker container through a container image; integrating the Gin application into the gateway service with a Gin middleware service, and using the gateway as an access point to the cloud platform, wherein the cloud platform architecture is K8S; forwarding all traffic in the gateway to the Gin middleware service by configuring Ingress rules, so that the gateway service has an audit function.
[0017] In certain embodiments of the first aspect of the present application, it also includes: providing a visual operation interface; obtaining the user's service request by collecting the user's operation instructions in the visual operation interface, wherein the service request of the service requester is sent based on the interaction mode of multiple custom standardized interfaces and the shared service interface corresponding to the upper-layer application; the interaction mode of the multiple custom standardized interfaces includes a message interaction mode, a publish-subscribe mode and a database access party; the user operation instructions determine the shared service interface corresponding to each upper-layer application according to the needs of the upper-layer application, including at least one of the following: based on the access rights corresponding to the third-party application, controlling whether the third-party application accesses the data sharing system, and providing a channel encryption mechanism; or, based on the access rights and access volume requirements of multiple tenants, determining the The available interface of each tenant in the multi-tenant system; or, based on the access frequency of the third-party application, determine whether to restrict the access of the third-party application; or, based on the priority order of the service requests in the access queue and the connection status of the service requests, adjust the processing priority of the service requests of different priorities; audit the service requests, and after the audit is passed; based on the load balancing algorithm and the cluster performance weights of different server clusters in the determined server cluster, select the target server cluster for processing the service request of the service requester, and the target server cluster returns the corresponding response data; based on the response data, provide the service requester with a visual interface to display the response data; the visual interface also includes an application release record interface and a system operation interface.
[0018] In certain embodiments of the first aspect of the present application, determining the audit log and storing it also includes: storing the audit log in the blockchain; if the service request passes the review of the audit middleware, obtaining the audit log related to the service request from the data storage node; wherein the audit log is an audit log package generated in accordance with the blockchain format, and the audit log package carries a signature.
[0019] The second aspect of the present application discloses an audit system based on the Gin framework cloud platform, including: a middleware generation module, which is used to respond to a service request for accessing the cloud platform, generate a corresponding type of audit middleware according to the type of the service request, and store the audit middleware in a gateway service, wherein the gateway service is constructed based on the Gin framework; an audit module, which is used to call the audit middleware to intercept the service request, audit the service request according to preset audit rules, determine the audit log and store it, wherein the preset audit rules include at least one of network settings, identity authentication, access rights and log records.
[0020] The third aspect of the present application discloses an electronic device, comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the above method.
[0021] The beneficial effects of the present application are as follows: by responding to a service request to access the cloud platform, generating a corresponding type of audit middleware according to the type of the service request, and storing the audit middleware in a gateway service, the gateway service is constructed based on the Gin framework; calling the audit middleware to intercept the service request, auditing the service request according to preset audit rules, determining the audit log and storing it, the preset audit rules include at least one of network settings, identity authentication, access rights and log records. By integrating all the middleware into the gateway with the Gin architecture, on the one hand, security audit capabilities are integrated; on the other hand, more flexible security management and control technologies are supported, and based on the Gin high concurrency framework, the cloud platform has the advantages of multi-process high concurrency, high-performance input and output, and fast exception handling. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 It is an application environment diagram of an audit system based on the Gin framework cloud platform in an embodiment of the present application;
[0023] Figure 2 It is a flowchart of an audit method based on the Gin framework cloud platform in an embodiment of the present application;
[0024] Figure 3 It is a logical architecture diagram of an audit method based on the Gin framework cloud platform in an embodiment of the present application;
[0025] Figure 4 It is a log collection and storage topology diagram in an audit based on the Gin framework cloud platform in an embodiment of the present application;
[0026] Figure 5 It is a flowchart of registering audit middleware in an audit based on the Gin framework cloud platform in an embodiment of the present application;
[0027] Figure 6 It is a service deployment diagram in an audit method based on the Gin framework cloud platform in an embodiment of the present application;
[0028] Figure 7 It is a complete architecture topology diagram of an audit method based on the Gin framework cloud platform in an embodiment of the present application;
[0029] Figure 8 It is a structural diagram of an audit system based on the Gin framework cloud platform in an embodiment of the present application;
[0030] Fig. 9 It is a structural schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION
[0031] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and sub-samples in the embodiments can be combined with each other without conflict.
[0032] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.
[0033] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.
[0034] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so as to describe the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.
[0035] Unless otherwise stated, the term "plurality" means two or more.
[0036] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.
[0037] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.
[0038] Combination Figure 1As shown, an implementation environment diagram of an audit system based on the Gin framework cloud platform provided by an embodiment of the present disclosure includes a user end and a server end, wherein the user end and the server end are connected to the network and communicate with each other. In this application, the user end is the front end, and the server end is the back end, that is, a Gin framework cloud platform audit system is deployed between the front end and the back end, which will not be repeated.
[0039] For example, the audit system based on the Gin framework cloud platform can be applied to audit access to the cloud platform in the terminal, can also be applied to audit the information responded by the cloud platform in the server, and can also be software running in the terminal or server for auditing. In some embodiments, the terminal can be at least one of a computer device, a desktop device, a smart phone, and a tablet device; the server can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and big data and artificial intelligence platforms; the software can be an audit application based on the Gin framework cloud platform, etc., but is not limited to the above forms.
[0040] Combination Figure 2 As shown, the embodiment of the present disclosure provides a flow chart of an audit method based on the Gin framework cloud platform, including:
[0041] Step S201, in response to a service request for accessing the cloud platform, generating audit middleware of a corresponding type according to the type of the service request, and storing the audit middleware in a gateway service, wherein the gateway service is constructed based on a Gin framework;
[0042] Among them, cloud platform is a computing method and service model based on the Internet, which allows users or enterprises to obtain the required computing resources (including servers, storage, databases, software, networks, etc.) and services through the network in an on-demand and easily scalable manner. For example, cloud platform can be divided into infrastructure as a service, platform as a service and software as a service by type, which will not be repeated here.
[0043] It should be noted that service requests are generated by users or the client where users are located. The types of service requests are diverse, for example, information service requests (query, consultation), system service requests (verification, confirmation), that is, different audit middleware is set according to the type of service request. One type of audit middleware can correspond to multiple different types of service requests, or one type of audit middleware can correspond to one type of service request.
[0044] It should also be noted that the gateway service (API gateway or microservice gateway) is a network architecture model that serves as an intermediary layer between the internal services of the system and the external clients. The gateway service is built based on the Gin framework, making the cloud platform embedded with the Gin framework lightweight, high-performance and easy to use. The Gin framework is a lightweight HTTP Web framework written in Go (Golang), which is highly respected in the field of Web development in the Go language for its high performance, concise API and rich functional features.
[0045] Specifically, when building a gateway service based on the Gin framework, the following functions can be achieved:
[0046] Routing management, identity authentication, current limiting and circuit breaking (current limiting of external requests to prevent malicious requests or sudden traffic from putting pressure on internal services; at the same time, implementing a circuit breaking mechanism to protect internal services from further deterioration in the event of a failure), protocol conversion, logging and monitoring.
[0047] In short, using the Gin framework to build gateway services has the advantages of being efficient, flexible, and scalable.
[0048] Optionally, in some embodiments, generating audit middleware of a corresponding type according to the type of the service request, and storing the audit middleware in the gateway service includes:
[0049] Build multiple middlewares based on the high-concurrency Gin framework, and sequentially organize the multiple middlewares into a middleware chain;
[0050] Determine the type of the service request, and determine the location information of a matching middleware in the middleware chain according to the type of the service request, wherein the service request is an Http request;
[0051] The processing audit logic of the middleware corresponding to the location information is configured by presetting a hook function, the configured middleware is determined as the audit middleware, and is cached to the gateway service of the local gateway.
[0052] Through the above method, it is possible to support the automatic generation of the audit middleware of the Gin framework, and cache the generated audit middleware to the gateway service of the local gateway, that is, to use it as soon as it is generated. On the one hand, it optimizes the data architecture; on the other hand, it also greatly reduces the amount of data on the audit platform.
[0053] Step S202, calling the audit middleware to intercept the service request, auditing the service request according to preset audit rules, determining and storing audit logs, wherein the preset audit rules include at least one of network settings, identity authentication, access rights and log records.
[0054] Specifically, the audit middleware is called to intercept the service request, and the user ID, operation type, timestamp, routing information, request message and response message of the intercepted service request are collected to form a temporary file, wherein the collection is performed through a Fluentd streaming data collector;
[0055] Audit and verify the network settings, identity authentication, access rights and log records in the temporary file one by one according to the preset audit rules to generate an audit log;
[0056] The audit log is processed into blocks to generate audit log blocks, and the audit log blocks are written into a back-end database, which is a distributed storage.
[0057] For example, the Gin framework is applied to the gateway service, and an HTTP (Hypertext Transfer Protocol) access interface is provided to the functional entity of this domain when performing SIP (Session Initiation Protocol) session control. Based on the Gin middleware, a middleware function is created to intercept requests, and the logging logic is inserted into the request processing flow according to the various security audit rules implemented. By registering this middleware to the Gin routing group, a security audit of all incoming requests can be implemented. Through the above method, various security audit logics can be conveniently processed centrally, and various security audit rules and functions can be customized and expanded as needed, such as network setting audit, identity authentication audit, access permission audit, and operation log record audit. In addition, the industry's mature Fluentd audit log collection and Elasticsearch / Loki distributed storage form a complete integrated solution. In this embodiment, the Gin framework is applied to the gateway service, which can make full use of the advantages of the Gin framework's high concurrency of multiple coroutines, high-performance input and output, and fast exception handling, improve the response efficiency and access speed of the gateway service, and enhance the robustness of the gateway service processing request Request. Finally, the audit middleware also implements its own visual interface, which more conveniently provides users with a series of functions such as security audit configuration, audit log retrieval and query.
[0058] In some of the above embodiments, after the audit log is determined and stored, the process further includes:
[0059] Performing data analysis on the audit log to generate an analysis report, and adjusting the audit strategy of the audit middleware based on the analysis report, wherein the analysis report includes abnormal data type, occurrence time, impact scope, abnormal cause and remedial measures;
[0060] If the requester who sends the service request in the analysis report is a user in a preset list, determining the level of the audit policy according to the identity information of the user, adjusting the audit policy according to the corresponding level and responding, the identity information at least including the user's position information;
[0061] If the request time of any of the service requests involved in the analysis report is within a preset time period, the audit policies of different levels are determined according to the time interval differences within the preset time period, and the audit policies are adjusted and responded to according to the corresponding levels.
[0062] Collect all relevant service request logs from the audit middleware, including request time, source, operation type, result and other information, and perform data cleansing to remove duplicate, invalid or incorrectly formatted data to ensure the accuracy and completeness of the analysis data.
[0063] Specifically, data analysis includes anomaly detection, time analysis, impact assessment, cause analysis, and remediation suggestions.
[0064] For example, anomaly detection uses data analysis tools (such as machine learning algorithms, statistical methods, etc.) to identify abnormal data types, such as unauthorized access, abnormal traffic surges, and sensitive data leakage.
[0065] For example, time analysis determines the time when abnormal events occur and analyzes whether they are concentrated in a specific period of time or show periodicity.
[0066] For example, impact assessment evaluates the scope of impact of abnormal events, including the affected systems, data volume, number of users, etc.
[0067] For example, cause analysis combines contextual information (such as user behavior, system status, etc.) to analyze the cause of the exception.
[0068] For example, remediation recommendations, based on the analysis results, propose specific remedial measures and preventive measures.
[0069] Combined with the data results of the above analysis data, an analysis report with a preset report template is generated, which is easy to read and understand. The preset report template includes charts, trend analysis, risk level assessment, etc.
[0070] In addition, check whether the requester in the analysis report is in the preset list. If so, determine the level of the audit policy based on the user's identity information (such as position information). For example, if the requester is a high-level user, it means that the user has higher authority and trust, and only moderate auditing is required to prevent internal threats; if the requester is a low-level user, auditing should be strengthened to monitor potential security risks.
[0071] In addition, audit strategies can be adopted according to different time periods. For example, during business peak periods (based on industry differences and visit volume), the level of auditing needs to be reduced to reduce the impact on business, but key operations can be monitored. For example, during non-peak periods, auditing efforts can be increased and more comprehensive reviews can be conducted.
[0072] Through the above method, the audit strategy can be adjusted automatically or manually according to the analysis results and different users / time periods, and corresponding response measures can be implemented; at the same time, the audit middleware can more effectively identify and respond to potential security risks, and flexibly adjust the audit strategy according to specific circumstances to ensure the security and stability of the system.
[0073] Optionally, in some embodiments, after the audit log is determined and stored, the process further includes:
[0074] Analyze the query request using a data analysis engine, and split the query request into multiple query sub-requests according to the analysis result;
[0075] Distributed search is used to process multiple query sub-requests in parallel, the audit logs corresponding to the query sub-requests are merged according to a preset log template, and a log feedback result is output.
[0076] Specifically, as part of the service request, when a query request is received, a data analysis engine (such as Apache Spark, PrestoDB, etc.) is used to analyze the query request, including but not limited to the target data set of the query, query conditions (such as time range, keywords, specific fields, etc.). Based on the analysis results, the complex query request is split into multiple simpler query sub-requests that are easier to process in parallel. For example, each sub-request should focus on one or more specific aspects of the original request to facilitate independent execution and result aggregation.
[0077] The split query sub-requests are assigned to different nodes or instances in the distributed search system (such as Elasticsearch, Solr, etc.) for processing. Each node is responsible for processing one or more sub-requests to achieve parallel computing. Each node executes the corresponding sub-request on the data shards it manages, including scanning indexes, applying filtering conditions, performing aggregation operations, etc. In a distributed system, each node will generate intermediate results, and each result will be deduplicated, sorted, merged, etc. to ensure the accuracy of the final result.
[0078] Through the above method, the data analysis engine is used to perform complex query analysis, and the query request is split into multiple sub-requests for distributed parallel processing, which can significantly improve the efficiency and response speed of the query; at the same time, the results are integrated and formatted through the preset log template, ensuring that users can easily obtain and understand the query results.
[0079] Optionally, in some embodiments, the gateway service is constructed based on the Gin framework, including:
[0080] Obtain a package management tool based on the development environment of the go language, and use the package management tool to install the Gin framework;
[0081] Generate an executable file with the Gin application by compiling the go code, create a Docker container, and deploy the executable file in the Docker container through the container image;
[0082] Integrate the Gin application into the gateway service with the Gin middleware service, and use the gateway as the access point of the cloud platform, wherein the cloud platform architecture is K8S;
[0083] By configuring the Ingress rules, all traffic in the gateway is forwarded to the Gin middleware service, so that the gateway service has an audit function.
[0084] Specifically, by using Go language package management tools (such as go get or go mod), it is helpful to obtain and manage the libraries and frameworks that the project depends on. For example, the Gin framework helps to maintain the consistency of project dependencies, reduce problems caused by version conflicts, and improve development efficiency.
[0085] Compiling Go code into executable files and deploying them through Docker containers enables applications to run in a consistent manner across environments. Since Docker containers provide a lightweight, portable application runtime environment, they reduce problems caused by differences in environment configurations and accelerate the deployment process of applications from development to production environments.
[0086] Integrate the Gin application as a middleware service into the gateway service and build a microservice architecture to decouple services and improve the scalability and maintainability of the system. As an access point to the cloud platform, the gateway can uniformly handle cross-cutting concerns such as request routing, security authentication, and current limiting, thereby improving the overall service quality and security.
[0087] Deploying applications on the Kubernetes (K8S) cloud platform and leveraging K8S's automatic deployment, automatic expansion, load balancing and other features can further improve application availability and scalability. At the same time, the declarative configuration and rich resource management tools provided by K8S make application operation and maintenance management more convenient and efficient.
[0088] By configuring Ingress rules, all traffic in the gateway is forwarded to the Gin middleware service, achieving fine-grained control and management of traffic. In addition, the Gin application is integrated into the gateway as a middleware service to conveniently implement audit functions at the service level, such as recording request logs and monitoring request behaviors, providing strong support for the security and compliance of the cloud platform audit system.
[0089] Through the above methods, through efficient package management, rapid containerized deployment, microservice integration, cloud platform automated management, and traffic and audit control, a complete solution is provided for application development, deployment, and operation and maintenance based on the Go language and Gin framework, which significantly improves development efficiency and the stability, scalability, and security of system operation.
[0090] Optionally, in some embodiments, it further includes:
[0091] Provide a visual operation interface;
[0092] The user's service request is obtained by collecting the user's operation instructions in the visual operation interface, wherein the service request of the service requester is sent based on the interaction modes of multiple customized standardized interfaces and the shared service interface corresponding to the upper-layer application; the interaction modes of the multiple customized standardized interfaces include message interaction mode, publish-subscribe mode and database access mode;
[0093] The user operation instruction determines the shared service interface corresponding to each upper-layer application according to the needs of the upper-layer application, including at least one of the following: based on the access authority corresponding to the third-party application, controlling whether the third-party application accesses the data sharing system and providing a channel encryption mechanism; or, based on the access rights and access volume requirements of multiple tenants, determining the available interface of each tenant in the multiple tenants; or, based on the access frequency of the third-party application, determining whether to restrict the access to the third-party application; or, based on the priority order of the service requests in the access queue and the connection status of the service requests, adjusting the processing priority of the service requests of different priorities;
[0094] Audit the service request, and wait for the audit to pass;
[0095] Based on the load balancing algorithm and the determined cluster performance weights of different server clusters in the server cluster, a target server cluster is selected to process the service request of the service requester, and the target server cluster returns corresponding response data; based on the response data, a visual interface is provided to the service requester to display the response data; the visual interface also includes an application release record interface and a system operation interface.
[0096] Among them, it should be noted that a visual operation interface is provided through a display screen, in which the user operation instructions in the visual operation interface are collected, the user's service request and the interaction method of the customized standardized interface are obtained, and at the same time, according to the user's operation instructions, the shared service interface to be used for the service request is obtained.
[0097] Through the above methods, the usability, security, flexibility and performance of the cloud platform audit system are improved by providing a visual operation interface, flexible customized standardized interface, refined permission and access control, efficient service request auditing and priority adjustment, as well as high-performance response based on load balancing and a comprehensive visual interface.
[0098] See also Figure 3 , is a logical architecture diagram of an audit method based on the Gin framework cloud platform in an embodiment of the present application, including:
[0099] As shown in the figure, in the K8S cluster, the nodes of the third-party application (APP service, daemon service), among which the daemon service generates audit logs and the APP service generates application logs, should be combined with the modules in the above figure to achieve effective request interception, audit method implementation, audit policy setting, audit data collection, storage, analysis and management, etc. In addition, the implementation of the plan should take into account factors such as the specific needs, budget and technical environment of the organization. It is important to ensure that all audit activities comply with relevant regulations and standards to ensure the data security and compliance of the organization.
[0100] Define audit requirements and clarify audit requirements. Common audit methods include: 1) Network setting audit, check whether the configuration of network equipment, firewalls and routers meets security requirements; 2) Identity authentication audit, audit the identity authentication policy of the cloud platform, and verify the legitimacy and security of user accounts; 3) Access permission audit, audit the user's operating permissions in the cloud platform to ensure that the allocation of permissions complies with security principles; 4) Log record audit, audit the operation log recording mechanism of the cloud platform, and check whether the log contains abnormal behavior.
[0101] Create audit middleware. The Gin framework allows developers to add their own hook functions in the process of processing requests. This hook function is called middleware. By creating a Gin middleware to handle the audit logic, this middleware can intercept all HTTP requests and check whether audit logs need to be recorded according to security audit requirements. It decides whether audit information needs to be recorded based on the characteristics of each request (such as URL, request method, user identity, request message, etc.).
[0102] For example, you could create a middleware function called AuditMiddleware as follows:
[0103] func AuditMiddleware()gin.HandlerFunc{
[0104] return func(c*gin.Context){
[0105] / / Audit logic code
[0106] / / Record audit information as needed
[0107] c.Next()
[0108] }
[0109] }
[0110] Audit log collection and storage, see Figure 4 , is a topological structure diagram of audit log collection and storage in an audit based on the Gin framework cloud platform in an embodiment of the present application; the details are as follows:
[0111] It should be noted that in this embodiment, Fluentd streaming audit log collection technology and Elasticsearch / Loki distributed search and analysis engines are integrated to implement a complete integrated audit log collection and storage solution. Among them, Fluentd is responsible for collecting and processing audit information from multiple data sources (temporary log files and distributed caches), and centrally storing them in Elasticsearch / Loki distributed storage to ensure that audit logs are easy to retrieve and analyze for review when needed.
[0112] Register the middleware and register the audit middleware with your Gin routing group. In this way, every request passing through the routing group will be processed by the audit middleware. You can use the gin.Use() function to register the middleware.
[0113] See also Figure 5 , which is a flow chart of registering audit middleware in an audit based on the Gin framework cloud platform in an embodiment of the present application. The user sends a registration request carrying a certain type of audit middleware. If the registration request is detected, the registration request is forwarded to the service process. By starting the Gin function, the audit middleware is added to the service corresponding to the gateway.
[0114] For example, in the main function, you can register the middleware like this:
[0115] router:=gin.Default()
[0116] router.Use(AuditMiddleware()) / / Register audit middleware
[0117] Policy configuration and dynamic adjustment: Configure audit policies based on audit requirements and allow dynamic adjustment of these policies. For example, different audit levels can be defined based on user roles or time periods, ensuring that the audit system is flexible enough to adapt to changing business needs and security requirements.
[0118] Test and optimize: Test your audit middleware to ensure it correctly records the required information and provides sufficient context when needed. Optimize and improve based on actual conditions to ensure the performance and reliability of the audit system.
[0119] Integration and deployment, in Figure 6 In the process, gin middleware is deployed to the gateway service of the cloud platform so that gin middleware can intercept all requests. The core architecture of the cloud platform is K8S. Therefore, the audit middleware service needs to be deployed after Ingress so that Ingress can send all its traffic to this service first. The gin framework supports multiple deployment methods, including Docker container deployment, cloud service deployment, etc. At the same time, gin's high performance makes it very suitable for use in production environments.
[0120] In this embodiment, Docker container deployment is used. The Dockerfile file content deployed this time defines two phases: the build phase and the run phase. In the build phase, the go code is compiled and an executable file is generated. In the run phase, a lightweight container is created and the executable file is copied to the container.
[0121] Operation visualization interface: Provide users with an intuitive visualization interface to facilitate viewing and analyzing audit data, as well as setting various security audit policies, so that all user operations and inter-system call requests will be intercepted by the audit middleware and the corresponding messages will be recorded for user query and analysis.
[0122] Optionally, see Figure 7 , a complete architecture topology diagram of an audit method based on the Gin framework cloud platform in an embodiment of the present application is described in detail as follows:
[0123] Users send service requests through mobile devices, PCs or other devices, and the service requests are transmitted to reverse proxies or dynamic and static separation for forwarding through load balancing. When the service request flows to the API gateway cluster with a unified traffic access point, it is audited by matching the audit middleware that matches the service request. High-performance audit middleware is implemented based on the gin framework, and the audit middleware is deployed between the gateway and the service. The gateway serves as an API aggregation point and unified access point for microservices scattered in various business systems. External requests can access all internal REST API services by accessing this access point. Then, an indirect middle layer is added behind the gateway to intercept all traffic requests passing through the gateway to meet security audit requirements. The audit middleware integrates mainstream security audit solutions and realizes a closed loop of business. It is precisely because of the abstraction of the security audit middle layer that redundant code and functions are greatly reduced, maintenance work is more unified and convenient, error is reduced, and performance is greatly improved.
[0124] In the embodiment of this city, the middleware is implemented based on the gin high-concurrency framework of the more powerful golang language, and the performance is extremely powerful. The middleware not only supports a variety of mainstream functions of security configuration auditing, such as network setting auditing, identity authentication auditing, access rights auditing, and operation log recording auditing, but also supports various protocols of major mainstream cloud platforms such as HTTP, HTTPS, etc., and supports docking with ES and Loki mainstream distributed storage. Even if some solutions of different development language frameworks themselves, such as the Java language can adopt Spring's AOP aspect asynchronous logging solution, it can effectively decouple the logging code and business logic code, and avoid logging blocking the main thread in an asynchronous way, thereby improving the response speed and throughput of the system. This solution is completely dependent on the development language framework itself, and the technology stack cannot be unified. This application provides a unified and independent cloud platform high-performance security auditing middleware, which effectively overcomes the above problems. In addition, since logging operations usually require IO operations, it will have a certain impact on the business response time. Through the visual operation interface, it is more conducive to the maintainability of the code.
[0125] See also Figure 8 , is a structural diagram of an audit system based on the Gin framework cloud platform in an embodiment of the present application, including:
[0126] The middleware generation module 801 is used to respond to the service request for accessing the cloud platform, generate the corresponding type of audit middleware according to the type of the service request, and store the audit middleware in the gateway service, which is constructed based on the Gin framework;
[0127] The audit module 802 is used to call the audit middleware to intercept the service request, audit the service request according to preset audit rules, determine the audit log and store it. The preset audit rules include at least one of network settings, identity authentication, access rights and log records.
[0128] It should be noted that the audit method based on the Gin framework cloud platform of this application corresponds one-to-one to the audit system based on the Gin framework cloud platform. The corresponding technical details, i.e., the technical solutions, refer to the above-mentioned audit method based on the Gin framework cloud platform, and will not be repeated here.
[0129] Through the above methods, the audit system based on the Gin framework cloud platform has the following technical effects: First, a unified middleware architecture is implemented, which can be seamlessly integrated into the mainstream cloud platform without changing the cloud platform; second, it seamlessly supports various protocols of major mainstream cloud platforms such as HTTP, HTTPS and other protocols; third, it supports more flexible management and control technologies, such as secondary authorization, to realize the blacklist, secondary authorization, and whitelist workflow of cloud administrators performing dangerous operations when the cloud platform is running, and provide audit logs of all operations of cloud administrators. Fourth, it supports a variety of mainstream security configuration audit functions, such as network setting audit, identity authentication audit, access rights audit, operation log record audit, etc. Fifth, based on the Gin high-concurrency framework with more powerful golang language, the performance cost is only about 100 milliseconds, which has little impact on user operations. Sixth, based on Fluentd streaming audit log collection technology, the Elasticsearch / Loki distributed search and analysis engine is used to realize the real-time collection and retrieval capabilities of application logs.
[0130] In other embodiments, the present disclosure also provides an electronic device, including: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the above method.
[0131] Fig. 9 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Fig. 9 The computer system 900 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0132] like Fig. 9As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 902 or the program loaded from the storage part 508 to the random access memory (RAM) 903, such as executing the method in the above embodiment. In the random access memory 903, various programs and data required for system operation are also stored. The central processing unit 901, the read-only memory 902 and the random access memory 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0133] The following components are connected to the input / output interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed so that a computer program read therefrom is installed into the storage section 908 as needed.
[0134] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 909, and / or installed from a removable medium 911. When the computer program is executed by a central processing unit (CPU) 901, various functions defined in the system of the present application are executed.
[0135] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent possible changes only. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. Parts and subsamples of some embodiments may be included in or replace parts and subsamples of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of a stated sub-sample, whole, step, operation, element, and / or component, but do not exclude the presence or addition of one or more other sub-samples, wholes, steps, operations, elements, components and / or groups of these. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.
[0136] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. Technicians can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0137] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some sub-samples can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.
[0138] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the method, system and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.
Claims
1. An audit method based on the Gin framework cloud platform, characterized in that: include: In response to a service request for accessing the cloud platform, generating audit middleware of a corresponding type according to the type of the service request, and storing the audit middleware in a gateway service, wherein the gateway service is constructed based on a Gin framework; The audit middleware is called to intercept the service request, the service request is audited according to preset audit rules, and the audit log is determined and stored. The preset audit rules include at least one of network settings, identity authentication, access rights and log records.
2. The method according to claim 1, characterized in that: Once the audit logs are determined and stored, this also includes: Performing data analysis on the audit log to generate an analysis report, and adjusting the audit strategy of the audit middleware based on the analysis report, wherein the analysis report includes abnormal data type, occurrence time, impact scope, abnormal cause and remedial measures; If the requester who sends the service request in the analysis report is a user in a preset list, determining the level of the audit policy according to the identity information of the user, adjusting the audit policy according to the corresponding level and responding, the identity information at least including the user's position information; If the request time of any of the service requests involved in the analysis report is within a preset time period, the audit policies of different levels are determined according to the time interval differences within the preset time period, and the audit policies are adjusted and responded to according to the corresponding levels.
3. The method according to claim 1 or 2, characterized in that: Generating a corresponding type of audit middleware according to the type of the service request, and storing the audit middleware in the gateway service, including: Build multiple middlewares based on the high-concurrency Gin framework, and sequentially organize the multiple middlewares into a middleware chain; Determine the type of the service request, and determine the location information of a matching middleware in the middleware chain according to the type of the service request, wherein the service request is an Http request; The processing audit logic of the middleware corresponding to the location information is configured by presetting a hook function, the configured middleware is determined as the audit middleware, and is cached to the gateway service of the local gateway.
4. The method according to claim 3, characterized in that Calling the audit middleware to intercept the service request, auditing the service request according to preset audit rules, determining and storing the audit log, including: Calling the audit middleware to intercept the service request, collecting the user ID, operation type, timestamp, routing information, request message and response message of the intercepted service request to form a temporary file, wherein the collection is performed through a Fluentd streaming data collector; Audit and verify the network settings, identity authentication, access rights and log records in the temporary file one by one according to the preset audit rules to generate an audit log; The audit log is processed into blocks to generate audit log blocks, and the audit log blocks are written into a back-end database, which is a distributed storage.
5. The method according to claim 4, characterized in that Once the audit logs are determined and stored, this also includes: Analyze the query request using a data analysis engine, and split the query request into multiple query sub-requests according to the analysis result; Distributed search is used to process multiple query sub-requests in parallel, the audit logs corresponding to the query sub-requests are merged according to a preset log template, and a log feedback result is output.
6. The method according to claim 1, characterized in that The gateway service is constructed based on the Gin framework, including: Obtain a package management tool based on the development environment of the go language, and use the package management tool to install the Gin framework; Generate an executable file with the Gin application by compiling the go code, create a Docker container, and deploy the executable file in the Docker container through the container image; Integrate the Gin application into the gateway service with the Gin middleware service, and use the gateway as the access point of the cloud platform, wherein the cloud platform architecture is K8S; By configuring the Ingress rules, all traffic in the gateway is forwarded to the Gin middleware service, so that the gateway service has an audit function.
7. The method according to claim 1, characterized in that Also includes: Provide a visual operation interface; The user's service request is obtained by collecting the user's operation instructions in the visual operation interface, wherein the service request of the service requester is sent based on the interaction modes of multiple customized standardized interfaces and the shared service interface corresponding to the upper-layer application; the interaction modes of the multiple customized standardized interfaces include message interaction mode, publish-subscribe mode and database access mode; The user operation instruction determines the shared service interface corresponding to each upper-layer application according to the needs of the upper-layer application, including at least one of the following: based on the access authority corresponding to the third-party application, controlling whether the third-party application accesses the data sharing system and providing a channel encryption mechanism; or, based on the access rights and access volume requirements of multiple tenants, determining the available interface of each tenant in the multiple tenants; or, based on the access frequency of the third-party application, determining whether to restrict the access to the third-party application; or, based on the priority order of the service requests in the access queue and the connection status of the service requests, adjusting the processing priority of the service requests of different priorities; Audit the service request, and wait for the audit to pass; Based on the load balancing algorithm and the determined cluster performance weights of different server clusters in the server cluster, a target server cluster is selected to process the service request of the service requester, and the target server cluster returns corresponding response data; based on the response data, a visual interface is provided to the service requester to display the response data; the visual interface also includes an application release record interface and a system operation interface.
8. The method according to claim 1, characterized in that Identify and store audit logs, including: The audit log is stored in the blockchain; if the service request passes the review of the audit middleware, the audit log related to the service request is obtained from the data storage node; wherein the audit log is an audit log package generated according to the blockchain format, and the audit log package carries a signature.
9. An audit system based on the Gin framework cloud platform, characterized in that: include: A middleware generation module, which is used to respond to a service request for accessing the cloud platform, generate a corresponding type of audit middleware according to the type of the service request, and store the audit middleware in a gateway service, wherein the gateway service is constructed based on the Gin framework; The audit module is used to call the audit middleware to intercept the service request, audit the service request according to the preset audit rules, determine the audit log and store it. The preset audit rules include at least one of network settings, identity authentication, access rights and log records.
10. An electronic device, characterized in that: include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method according to any one of claims 1 to 8.
Citation Information
Cited By
Multi-cloud unified management platform oriented middleware log auditing method and device
CN121841961A
