System login and operation log asynchronous recording method and system based on message queue

Abstract

The invention discloses a system login and operation log asynchronous recording method and system based on a message queue, belongs to the technical field of data security, and aims to solve the technical problem of how to realize security monitoring of login operation behaviors across platforms and among different databases. Comprising the steps that each to-be-monitored platform or system serves as a source end, an asynchronous operation thread is introduced for each source end, login and service operation of the source ends are recorded through the asynchronous operation threads, log records are formed, and the content of the log records comprises login information, operation behaviors and data changes before and after operation; a producer reads a log record of a source end, converts the content of the log record into a uniform format, and sends the content to a message queue; consuming the log record in the message queue through a consumer, and sending the log record to a monitoring platform; and checking and analyzing the log record of each source end through the monitoring platform, and providing an alarm service based on an analysis result.

Description

Technical Field

[0001] The present invention relates to the technical field of data security, and in particular to an asynchronous recording method and system for system login and operation logs based on a message queue. Background Art

[0002] In modern information systems, it is often necessary to record user logins or operations such as addition, deletion, modification, and query of sensitive system data, which facilitates users (administrators) to independently locate problems and trace data during use. System sensitive data generally includes: user login information, platform information management modules, system management modules, etc. (parts involving operations that require recording field changes and usage, including addition, deletion, and modification).

[0003] In addition, when it comes to logging across platforms or between heterogeneous databases, compatibility and reliability have become new challenges.

[0004] How to achieve secure monitoring of login operation behaviors across platforms and different databases is a technical problem that needs to be solved. Summary of the Invention

[0005] The technical task of the present invention is to address the above deficiencies and provide an asynchronous recording method and system for system login and operation logs based on a message queue to solve the technical problem of how to achieve secure monitoring of login operation behaviors across platforms and different databases.

[0006] In a first aspect, the present invention provides an asynchronous recording method for system login and operation logs based on a message queue, which is used to monitor logins and operations of different platforms or systems. The method includes the following steps:

[0007] Regarding each platform or system to be monitored as a source end, for each source end, an asynchronous operation thread is introduced, and the login and business operations of the source end are recorded through the asynchronous operation thread to form a log record. The content of the log record includes login information, operation behaviors, and data changes before and after the operation;

[0008] The producer monitors each source end in real time, reads the log records of the source end, converts the content of the log records into a unified format, and sends it to the message queue;

[0009] The consumer consumes the log records in the message queue and sends the log records to the monitoring platform;

[0010] The monitoring platform views and analyzes the log records of each source end and provides an alarm service based on the analysis results.

[0011] Preferably, the producer reads the log records at the source end, converts the log records into JSON format or other lightweight serialization formats to represent the content of the log records, and sends the converted log records to the message queue.

[0012] Preferably, the producer encrypts the converted log records to obtain encrypted log records, and sends the encrypted log records to the message queue;

[0013] Correspondingly, the consumer reads the encrypted log records from the message queue, decrypts them, and sends the decrypted log records to the monitoring platform.

[0014] Preferably, the operation behaviors include insert, update, and delete operations on login information and business data.

[0015] Preferably, for each source end, a predefined warning strategy is configured in the monitoring platform. Based on the log records corresponding to the source end, the log records are analyzed by a log analysis tool and the warning strategy to detect abnormal situations of the source end. Based on the analysis results, alarm information is generated and pushed to the corresponding administrator;

[0016] Among them, the pushing methods of the alarm information include text messages, emails, and message center reminders in the source end.

[0017] In a second aspect, an asynchronous logging system for system login and manipulation based on a message queue according to the present invention is used to monitor the login and operations of different platforms or systems by an asynchronous logging method for system login and manipulation based on a message queue as described in any item of the first aspect. The system includes an asynchronous operation module, a producer, a message queue, a consumer, and a monitoring platform;

[0018] Each platform or system to be monitored is used as a source end. An asynchronous operation module is deployed in each source end, and an asynchronous operation thread is introduced in the asynchronous operation module. The asynchronous operation module is used to record the login and business operations of the source end through the asynchronous operation thread to form log records. The content of the log records includes login information, operation behaviors, and data changes before and after the operations;

[0019] The producer is used to monitor each source end in real time, read the log records of the source end, and send the content of the log records to the message queue after converting them into a unified format;

[0020] The consumer is used to consume the log records in the message queue and send the log records to the monitoring platform;

[0021] The monitoring platform is used to view and analyze the log records of each source end and provide an alarm service based on the analysis results.

[0022] Preferably, the producer reads the log records at the source end, converts the log records into JSON format or other lightweight serialization formats to represent the content of the log records, and sends the converted log records to the message queue.

[0023] Preferably, the producer encrypts the converted log records to obtain encrypted log records, and sends the encrypted log records to the message queue;

[0024] Correspondingly, the consumer reads the encrypted log records from the message queue, decrypts them, and sends the decrypted log records to the monitoring platform.

[0025] Preferably, the operation behaviors include insert, update, and delete operations on login information and business data.

[0026] Preferably, for each source end, predefined warning strategies are configured in the monitoring platform. The monitoring platform analyzes the log records based on the log records corresponding to the source end, through a log analysis tool and the warning strategies, to detect abnormal situations of the source end, generates alarm information based on the analysis results, and pushes the alarm information to the corresponding administrator;

[0027] Among them, the pushing methods of the alarm information include SMS, email, and reminder in the message center of the source end.

[0028] The asynchronous logging method and system for system login and manipulation based on message queue of the present invention have the following advantages: for each system or platform to be monitored, the login and business operations at the source end are recorded through an asynchronous operation thread, the log records of each system or platform are sent to the message queue by the producer, the log records are consumed from the message queue by the consumer and sent to the monitoring platform, and the login and operations of each system or platform are monitored by the monitoring platform, improving the reliability and stability of data transmission, reducing the pressure on the source end system, not affecting normal business operations, supporting multiple types of databases, being easy to integrate with existing systems, simplifying the configuration process, reducing the maintenance cost, strengthening the security protection, and protecting sensitive information from being leaked. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0030] The present invention will be further described below with reference to the drawings.

[0031] Figure 1Flow chart of a method for asynchronous recording of system login and operation logs based on message queue in Embodiment 1. Specific embodiments

[0032] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present invention and be able to implement it. However, the embodiments cited are not intended to limit the present invention. Without conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0033] The embodiments of the present invention provide a method and system for asynchronous recording of system login and operation logs based on message queue, which are used to solve the technical problem of how to implement security monitoring of login operation behaviors across platforms and different databases.

[0034] Embodiment 1:

[0035] A method for asynchronous recording of system login and operation logs based on message queue in the present invention is used to monitor the login and operations of different platforms or systems. The method includes the following steps:

[0036] Step S100: Each platform or system to be monitored is used as a source end. For each source end, an asynchronous operation thread is introduced, and the login and business operations of the source end are recorded through the asynchronous operation thread to form a log record. The content of the log record includes login information, operation behaviors, and data changes before and after the operation;

[0037] Step S200: Each source end is monitored in real time by a producer, the log record of the source end is read, and after converting the content of the log record into a unified format, it is sent to the message queue;

[0038] Step S300: The log record in the message queue is consumed by a consumer, and the log record is sent to the monitoring platform;

[0039] Step S400: The log records of each source end are viewed and analyzed by the monitoring platform, and an alarm service is provided based on the analysis results.

[0040] In this embodiment, the operation behaviors include insert, update, and delete operations on login information and business data.

[0041] In step S100 of this embodiment, the log record of the source end is read by the producer, the content of the log record is converted into a JSON format or other lightweight serialization format to represent the log record, and the converted log record is sent to the message queue.

[0042] For each source, predefined warning strategies are configured in the monitoring platform. Based on the log records corresponding to the source, the log records are analyzed by a log analysis tool and the warning strategies to detect abnormal conditions of the source. Alarm information is generated based on the analysis results and pushed to the corresponding administrator. Among them, the pushing methods of the alarm information include text messages, emails, and reminders in the message center of the source.

[0043] As an improvement to the method of this embodiment, the producer encrypts the converted log records to obtain the encrypted log records, and sends the encrypted log records to the message queue. Correspondingly, the consumer reads the encrypted log records from the message queue, decrypts them, and sends the decrypted log records to the monitoring platform.

[0044] The method of this embodiment introduces a message queue as an intermediary, and passes the log records to the monitoring platform through the message queue, reducing the burden brought by direct connection, and at the same time improving the reliability and scalability of the transmission.

[0045] The prevention of this embodiment introduces asynchronous processing. On the basis of not affecting the main thread operation, asynchronous operation processing is introduced, and the behavior of recording logs is placed in an asynchronous thread. In the asynchronous thread, login information or data changes before and after operations are sent to the message queue Kafka.

[0046] The method of this instance introduces phased processing. A producer is introduced to monitor changes in the source database, convert them into a standardized message format and send them to the message queue. In various situations, the consumer is responsible for reading messages from the queue and performing corresponding insert, update, or delete operations.

[0047] Support for multiple types of databases: A message queue is provided as an adaptation layer to support various common relational and non-relational databases, ensuring the generality and compatibility of the system.

[0048] Enhanced security measures: Encrypt the log records sent to the message pair, and use encryption technology to ensure the security of data during transmission.

[0049] Comprehensive monitoring system: Through the monitoring platform, the diary records of each source are provided for viewing, analysis, and alarming, helping the administrators of each source quickly locate the fault points and take corresponding measures.

[0050] Embodiment 2:

[0051] An asynchronous log recording system for system login and manipulation based on a message queue according to the present invention includes an asynchronous operation module, a producer, a message queue, a consumer, and a monitoring platform.

[0052] Each platform or system to be monitored is used as a source end. An asynchronous operation module is deployed in each source end, and an asynchronous operation thread is introduced in the asynchronous operation module. The asynchronous operation module is used to record the login and business operations of the source end through the asynchronous operation thread to form a log record. The content of the log record includes login information, operation behaviors, and data changes before and after the operation.

[0053] In this embodiment, the operation behaviors include insert, update, and delete operations on login information and business data. The producer is used to read the log records of the source end, convert the log records into a JSON format or other lightweight serialization formats to represent the content of the log records, and send the converted log records to the message queue.

[0054] The producer is used to monitor each source end in real time, read the log records of the source end, convert the content of the log records into a unified format, and then send them to the message queue.

[0055] The consumer is used to consume the log records in the message queue and send the log records to the monitoring platform.

[0056] The monitoring platform is used to view and analyze the log records of each source end and provide an alarm service based on the analysis results.

[0057] In this embodiment, for each source end, a predefined warning strategy is configured in the monitoring platform. Based on the log records corresponding to the source end, the log records are analyzed by a log analysis tool and the warning strategy to detect abnormal situations of the source end. Alarm information is generated based on the analysis results and pushed to the corresponding administrator; among them, the pushing methods of the alarm information include text messages, emails, and message center reminders in the source end.

[0058] As an improvement to the system of this embodiment, the producer encrypts the converted log records to obtain encrypted log records, and sends the encrypted log records to the message queue; correspondingly, the consumer reads the encrypted log records from the message queue and decrypts them, and sends the decrypted log records to the monitoring platform.

[0059] The system of this embodiment can monitor the logins and operations of different platforms or systems through the method disclosed in Embodiment 1.

[0060] The above has introduced in detail the method and system for asynchronous recording of system login and operation logs based on message queues. In this article, specific examples are used to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. An asynchronous recording method for system login and operation logs based on a message queue, characterized in that For monitoring the logins and operations of different platforms or systems, the method includes the following steps: Regarding each platform or system to be monitored as a source end, for each source end, an asynchronous operation thread is introduced, and the asynchronous operation thread is used to record the logins and business operations of the source end to form log records. The content of the log records includes login information, operation behaviors, and data changes before and after the operations; The producer monitors each source end in real time, reads the log records of the source end, converts the content of the log records into a unified format, and then sends it to the message queue; The consumer consumes the log records in the message queue and sends the log records to the monitoring platform; The monitoring platform views and analyzes the log records of each source end and provides an alarm service based on the analysis results.

2. The asynchronous recording method of system login and operation logs based on a message queue according to claim 1, characterized in that The producer reads the log records of the source end, converts the log records into a JSON format or other lightweight serialization format to represent the content of the log records, and sends the converted log records to the message queue.

3. The asynchronous recording method for system login and operation logs based on a message queue according to claim 1, characterized in that The producer encrypts the converted log records to obtain encrypted log records and sends the encrypted log records to the message queue; Correspondingly, the consumer reads the encrypted log records from the message queue, decrypts them, and sends the decrypted log records to the monitoring platform.

4. The asynchronous recording method for system login and operation logs based on a message queue according to claim 1, wherein, The operation behaviors include insert, update, and delete operations on the login information and business data.

5. The asynchronous recording method for system login and operation logs based on a message queue according to claim 1, characterized in that For each source end, predefined warning strategies are configured in the monitoring platform. Based on the log records corresponding to the source end, the log records are analyzed by a log analysis tool and the warning strategies to detect abnormal situations of the source end. Based on the analysis results, alarm information is generated and the alarm information is pushed to the corresponding administrator; Among them, the pushing methods of the alarm information include text messages, emails, and reminders in the message center of the source end.

6. An asynchronous logging system for system login and operation logs based on message queues, characterized in that, For monitoring the logins and operations of different platforms or systems by using a method for asynchronous recording of system logins and manipulations based on a message queue as described in any one of claims 1-5, the system includes an asynchronous operation module, a producer, a message queue, a consumer, and a monitoring platform; Regarding each platform or system to be monitored as a source end, an asynchronous operation module is deployed in each source end. An asynchronous operation thread is introduced in the asynchronous operation module. The asynchronous operation module is used to record the logins and business operations of the source end through the asynchronous operation thread to form log records. The content of the log records includes login information, operation behaviors, and data changes before and after the operations; The producer is used to monitor each source end in real time, read the log records of the source end, convert the content of the log records into a unified format, and then send it to the message queue; The consumer is used to consume the log records in the message queue and send the log records to the monitoring platform; The monitoring platform is used to view and analyze the log records of each source end and provide an alarm service based on the analysis results.

7. The asynchronous logging system for system login and manipulation logs based on a message queue according to claim 6, wherein The producer is used to read the log records of the source end, convert the log records into a JSON format or other lightweight serialization format to represent the content of the log records, and send the converted log records to the message queue.

8. The asynchronous logging system for system login and manipulation logs based on a message queue according to claim 6, characterized in that, The producer is used to encrypt the converted log records to obtain the encrypted log records, and send the encrypted log records to the message queue; Correspondingly, the consumer is used to read the encrypted log records from the message queue, decrypt them, and send the decrypted log records to the monitoring platform.

9. The asynchronous logging system for system login and manipulation logs based on a message queue according to claim 6, wherein The operation behaviors include insertion, update, and deletion operations on login information and business data.

10. The asynchronous logging system for system login and manipulation logs based on a message queue according to claim 6, characterized in that, For each source end, predefined warning strategies are configured in the monitoring platform. The monitoring platform is used to analyze the log records based on the log records corresponding to the source end, through a log analysis tool and the warning strategies, to detect abnormal situations of the source end, generate alarm information based on the analysis results, and push the alarm information to the corresponding administrator; Among them, the pushing methods of the alarm information include text messages, emails, and reminders in the message center of the source end.

Images