Cache data visualization system and method

Abstract

The invention discloses a cache data visualization system and method, and the system comprises an information aggregation module which integrates network environment redis clusters, monitoring graphs, alarm messages and container information, and returns the integrated information to a front end in a centralized manner through an agent; the authority control module is used for distributing ACL rules online and supporting on-demand combination of all authority commands of the original redis; the cluster data management module is used for adding, deleting, modifying and checking all data of any right redis cluster on line, maintaining the data in a point-to-point mode, and also comprises manual master-slave switching; according to the audit log white list module, the front end configures which command users are allowed to execute and which command users are allowed to not execute; and the integrated display module is integrated with the front end, provides a development API and displays a result. According to the scheme, the learning cost of the user is reduced, and the user can use and manage the redis cluster conveniently in a data visualization mode.

Description

Technical Field

[0001] The present invention relates to the technical field of software operation and maintenance, and particularly relates to a cache data visualization system and method. Background Art

[0002] Distributed cache - Redis: Redis (Remote Dictionary Server) is an open - source, high - performance key - value storage system. It can be used not only as a database, cache, and message middleware, but also, due to its characteristics, is particularly suitable for caching in a distributed environment. The following are some key points on how to use Redis as a cache in a distributed system: 1. High performance and speed Redis is memory - based, which means that read and write operations are much faster than disk - based storage systems. This makes Redis an ideal choice for fast caching in a distributed system.

[0003] 2. Data persistence Although Redis mainly relies on memory, it also provides data persistence mechanisms such as RDB snapshots and AOF (Append Only File) logs to prevent data loss.

[0004] 3. Clustering and replication Redis supports master - slave replication and cluster modes. In master - slave replication, one or more slave servers can replicate data from the master server. This can be used not only for read - write separation to improve read performance but also as a backup. In the cluster mode, data is distributed across multiple nodes, providing higher availability and load balancing.

[0005] 4. Message queue and publish / subscribe Redis can also be used as a message middleware to implement message queues and event - driven architectures through list data types and publish / subscribe functions.

[0006] For Redis, it is a server - side application, and all operations are performed in the black screen, that is, after commands. This causes a lot of inconvenience to managers / users: Learning command - line parameters requires a large time cost.

[0007] Managing Redis is not convenient. There are strict permission controls within enterprises. One needs to log in to the bastion host and then log in to the container.

[0008] All data is in the black screen, and users cannot intuitively see certain data content.

[0009] Manual operation in the black screen is prone to errors.

[0010] Multiple maintenance is required across clusters and networks, and the entrances are not unified.

[0011] Information is scattered and not aggregated. Monitoring needs to be done on the monitoring website, containers need to be sent to the container cloud, and alarms need to be sent to the alarm center.

[0012] Therefore, a cache data visualization solution is needed that is convenient for users to use and manage redis clusters. Summary of the invention

[0013] In response to the above technical problems, the present invention provides a cache data visualization system and method.

[0014] The present invention is implemented by adopting the following technical scheme: a cache data visualization system, comprising: Information aggregation module: integrates the redis clusters, monitoring graphs, alarm messages and container information of each network environment, and returns them to the front end through the proxy; Permission control module: online allocation of ACL rules, supporting on-demand combination of all native redis permission commands; Cluster data management module: online addition, deletion, modification and query of all data in any authorized redis cluster, maintaining data in a point-to-point manner, and also including manual master-slave switching; Audit log whitelist module: The front end configures which commands users are allowed to execute and which commands they are not allowed to execute; Integrated display module: Integrates with the front end, provides development API, and displays results.

[0015] Specifically, the information aggregation module deploys a set of proxy services in each environment, and the front end calls the corresponding environment proxy service according to different environment tags; the proxy service obtains the redis cluster interface, alarm information interface and container resource interface of its own environment according to the parameters passed by the front end.

[0016] Specifically, the permission control module calls the proxy service ACL interface with the help of the proxy service. The ACL interface obtains user permission information from the configuration dictionary configmap of k8s. The configuration dictionary configmap stores all authentication information and is mounted into the container by the proxy service. When changes are detected, it is automatically added to the proxy service itself.

[0017] Specifically, the data management of the cluster data management module includes: For redis cluster data white screen operation, use the encapsulated redis console interface service provided by the backend to process various addition, deletion, modification and query requests sent by the frontend; The console interface is directly connected to the Redis cluster. Through the RESP protocol and encapsulated requests, the requests are directly sent to the corresponding nodes for operation; the encapsulated requests are automatically recognized according to the Redis deployment mode. If the Redis master-slave is a single-node connection, the Redis cluster is a multi-node cluster connection, or the Redis Sentinel is a sentinel cluster connection, a unified interface is provided to implement data operations in the corresponding mode.

[0018] Specifically, the audit log whitelist module audits all operations of all users on the white screen and records the operation types that need to be logged. The audit levels include: Interface-level audit called on the page; The whitelist configured with front-end executable commands is filtered. Those within the whitelist are allowed to execute, otherwise an error message is prompted.

[0019] Specifically, the integrated display module provides corresponding interfaces for all places related to data. For the data displayed on the front end, a corresponding interface service is set up to implement it, and the visualization requirements of the entire Redis are disassembled in a modular and microservices manner.

[0020] A method for visualizing cached data includes the following steps: Connect to the Redis function menu through the command line or the front-end page; Global overview, distinguishing the role types, and visually viewing the user side and the management side respectively.

[0021] Specifically, the visualization on the user side includes: The user imports through other modules to view the existing Redis cluster list; After selecting an instance, enter the instance details page to view the DB databases supported by the current instance and freely switch. The cluster mode is set with 1 DB database, and the master-slave mode is set with 16 DB databases; After selecting the DB database, the user can perform online addition, deletion, modification, and query of data. When querying data, different display formats are supported.

[0022] Specifically, the visualization on the management side includes: Daily maintenance by the Redis administrator, authorizing users' permissions for Redis; Support for Redis hot update configuration, updating the server-side parameters without restarting the service; Support for data migration, setting advanced configuration items to migrate the Redis cluster to other clusters; Support for viewing audit logs, recording every operation of users.

[0023] The beneficial effects of the present invention are as follows: The present invention reduces the learning cost of users and facilitates the use and management of the redis cluster in a data visualization manner. The method has the following advantages: Low learning cost, changing the black screen operation to a white screen operation; Flexible permission control; Automated background operation, reducing human errors; Information aggregation, aggregating middleware resources, monitoring resources, and statistical information and displaying them to users in a centralized manner, reducing operation time. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the structures shown in these drawings.

[0025] Figure 1 It is a schematic diagram of the cache data visualization method in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0027] It should be noted that: Similar reference numerals and letters indicate similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0028] The following combines the attached Figure 1 , and details some embodiments of the present invention. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0029] The present invention provides a cache data visualization system and method. In the first aspect, a cache data visualization system includes: Information aggregation module: integrating redis clusters, monitoring graphs, alarm messages, and container information in each network environment into one, and centrally returning them to the front end through a proxy; Permission control module: online allocating ACL rules, supporting the on-demand combination of all native redis permission commands; Cluster data management module: online addition, deletion, modification and query of all data in any authorized redis cluster, maintaining data in a point-to-point manner, and also including manual master-slave switching; Audit log whitelist module: The front end configures which commands users are allowed to execute and which commands they are not allowed to execute; Integrated display module: Integrates with the front end, provides development API, and displays results.

[0030] In this embodiment, the information aggregation module deploys a set of proxy services in each environment, and the front end calls the corresponding environment proxy service according to different environment labels; the proxy service obtains the redis cluster interface, alarm information interface and container resource interface of its own environment according to the parameters passed by the front end.

[0031] In this embodiment, the permission control module calls the proxy service ACL interface with the help of the proxy service. The ACL interface obtains user permission information from the configuration dictionary configmap of k8s. The configuration dictionary configmap saves all authentication information and is mounted into the container by the proxy service. When changes are detected, it is automatically added to the proxy service itself.

[0032] In this embodiment, the data management of the cluster data management module includes: For redis cluster data white screen operation, use the encapsulated redis console interface service provided by the backend to process various addition, deletion, modification and query requests sent by the frontend; The console interface is directly connected to the redis cluster, and the request is sent directly to the corresponding node for operation through the RESP protocol and the encapsulation request; the encapsulation request is automatically identified according to the redis deployment mode; If the redis master-slave is a single-node connection, the redis cluster is a multi-node cluster connection, or the redis sentinel is a sentinel cluster connection, a unified interface is provided to implement data operations in the corresponding mode.

[0033] In this embodiment, the audit log whitelist module audits all users' operations on the white screen and records the types of operations that need to be logged. The audit levels include: Interface-level auditing of calls on the page; The front-end executable commands are filtered by the whitelist configuration. Commands in the whitelist are allowed to be executed, otherwise an error message is displayed.

[0034] In this embodiment, the integrated display module provides corresponding interfaces for all data-related places, and sets a corresponding interface service to implement the data displayed on the front end, using modularization and microservices to disassemble the visualization requirements of the entire redis.

[0035] The present invention also proposes a cache data visualization method accordingly. Figure 1 As shown, the following steps are included: Connect to the redis function menu through the command line or front-end page; Global overview, differentiate between role types, and visualize the user side and management side separately.

[0036] In this embodiment, the visualization on the user side includes: Users can import other modules to view the list of existing redis clusters; After selecting an instance, enter the instance details page to view the DB databases supported by the current instance and switch them freely. The cluster mode is set with 1 DB database, and the master-slave mode is set with 16 DB databases. After selecting the DB database, users can add, delete, modify and query data online. When querying data, different display formats are supported.

[0037] In this embodiment, the visualization on the management side includes: Redis administrator daily maintenance, authorizing users to have access to redis; Support redis hot update configuration, update server parameters without restarting the service; Support data migration, set advanced configuration items, and migrate redis clusters to other clusters; Supports audit log viewing to record every operation of the user.

[0038] In one embodiment, a method for visualizing cached data includes the following functional modules: Information aggregation: collects redis clusters, monitoring graphs, alarm messages, and container information of each network environment, and returns them to the front end through the proxy; Flexible permission control: ACL rules can be flexibly assigned online, and all permission commands of native redis can be combined on demand; Cluster data management: online addition, deletion, modification and query of all data in any authorized redis cluster, data maintenance in a point-to-point manner, including manual master-slave switching; Audit log whitelist: The front-end can configure which commands users can execute and which commands they cannot execute; Integrated display: Integrate with the front end, provide development API, and display results.

[0039] The specific implementation of each module is as follows: 1. Information aggregation: deploy a set of proxy services in each environment, and the front end calls the corresponding environment proxy service according to different environment tags. The proxy service obtains the redis cluster interface, alarm information interface, container resource interface, etc. of its own environment according to the parameters passed by the front end.

[0040] A proxy layer is used here to perform a forwarding operation. Each proxy is only responsible for forwarding requests in its own environment, and the request will be transmitted to the corresponding service.

[0041] 2. Flexible permission control: It mainly transfers the ACL capability to white-screen operation, and also uses the proxy service to call the proxy service ACL interface. The interface obtains user permission information from the configuration dictionary (configmap) in k8s. All authentication information exists in the configuration dictionary, which is mounted into the container by the proxy service. Changes are detected and automatically added to the proxy service itself.

[0042] The ACL is not directly configured on the redis server. First, there is only one copy of the data. If each redis node stores it, it is easy to cause data inconsistency. Second, the proxy layer has more flexible permission configuration, such as DB-level access control, which is not supported by the redis core. 3. Cluster data management: For white-screen operations on data in redis, the backend will provide a packaged redis console interface service to handle various add, delete, modify and query requests sent by the frontend.

[0043] The console interface will directly connect to the redis cluster, encapsulate the request through the RESP protocol, and send the request directly to the corresponding node for operation. The encapsulated request here will be automatically identified according to the redis deployment mode. For example, the redis master-slave is a single-node connection, the redis cluster is a multi-node cluster connection, and the redis sentinel is a sentinel cluster connection. A unified interface is provided to implement data operations in the corresponding mode.

[0044] 4. Audit log whitelist: There are two levels of audit for all user operations on the white screen. The first is the interface-level audit called on the page, including each operator, interface parameters, etc. The second is which commands can be executed on the front end. There is a configurable whitelist to filter them. For example, commands such as shutdown, flushall, flushdb, and config cannot be executed. Commands in the whitelist are allowed to be executed, otherwise an error message will be displayed. The third is the ability to record the types of operations you want to log. For example, sensitive commands such as del must be logged, which is very important for later troubleshooting.

[0045] 5. Integrated display: Corresponding interfaces are provided for all data-related places. A corresponding interface service is developed to implement what data the front end needs to display. The modular and microservice approach is used to disassemble the visualization requirements of the entire redis, which is easy to implement and convenient to expand.

[0046] like Figure 1As shown, the global overview displays the resource status that users and administrators care about, the trend chart of the number of redis clusters accessed per week, the top several with the largest memory in the redis cluster, and the top several with the largest data volume in the redis cluster.

[0047] Display on the user side: Users can view the existing list of redis clusters, which is imported by other modules; After selecting an instance, enter the instance details page, where you can view which DBs (databases) the current instance supports and can freely switch. There is only 1 DB in the cluster mode and 16 in the master-slave mode; After selecting a DB, users can add, delete, modify, and query data online; When querying data, different display formats are supported, such as Json, Text, and Unicode.

[0048] Display on the management side: The management side is mainly for the daily maintenance of redis administrators; Authorize which users have what kind of permissions for a certain set of redis; Support hot update configuration of redis. Without restarting the service, update the server parameters; Support data migration. Optional advanced configuration items can be used to migrate a certain set of redis clusters to other clusters; Support viewing audit logs, recording every operation of users, for troubleshooting and accountability; Audit log whitelist. In most scenarios, the audit is concerned with abnormal operation records.

[0049] For the foregoing embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification belong to preferred embodiments, and the actions involved are not necessarily essential to the present application.

[0050] In the above embodiments, the basic principles, main features, and advantages of the present invention are described. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principles of the present invention. Without departing from the spirit and scope of the present invention, any changes and modifications made by those skilled in the art without departing from the spirit and scope of the present invention shall fall within the protection scope of the appended claims of the present invention.

Claims

1. A cache data visualization system, characterized in that, include: Information aggregation module: integrates the redis clusters, monitoring graphs, alarm messages and container information of each network environment, and returns them to the front end through the proxy; Permission control module: online allocation of ACL rules, supporting on-demand combination of all native redis permission commands; Cluster data management module: online addition, deletion, modification and query of all data in any authorized redis cluster, maintaining data in a point-to-point manner, and also including manual master-slave switching; Audit log whitelist module: The front end configures which commands users are allowed to execute and which commands they are not allowed to execute; Integrated display module: Integrates with the front end, provides development API, and displays results.

2. A cache data visualization system according to claim 1, characterized in that, The information aggregation module deploys a set of proxy services in each environment, and the front end calls the corresponding environment proxy service according to different environment tags; the proxy service obtains the redis cluster interface, alarm information interface and container resource interface of its own environment according to the parameters passed by the front end.

3. A cache data visualization system according to claim 1, characterized in that, The permission control module calls the proxy service ACL interface with the help of the proxy service. The ACL interface obtains user permission information from the configuration dictionary configmap of k8s. The configuration dictionary configmap saves all authentication information and is mounted into the container by the proxy service. Changes are detected and automatically added to the proxy service itself.

4. A cache data visualization system according to claim 1, characterized in that, The data management of the cluster data management module includes: For redis cluster data white screen operation, use the encapsulated redis console interface service provided by the backend to process various addition, deletion, modification and query requests sent by the frontend; The console interface is directly connected to the redis cluster, and the request is sent directly to the corresponding node for operation through the RESP protocol and the encapsulation request; the encapsulation request is automatically identified according to the redis deployment mode; If the redis master-slave is a single-node connection, the redis cluster is a multi-node cluster connection, or the redis sentinel is a sentinel cluster connection, a unified interface is provided to implement data operations in the corresponding mode.

5. A cache data visualization system as claimed in claim 1, wherein The audit log whitelist module audits all user operations on the white screen and records the types of operations that need to be logged. The audit levels include: Interface-level auditing of calls on the page; The front-end executable commands are filtered by the whitelist configuration. Commands in the whitelist are allowed to be executed, otherwise an error message is displayed.

6. The cache data visualization system according to claim 1, characterized in that, The integrated display module provides corresponding interfaces for all data-related places, and sets a corresponding interface service to implement the data displayed on the front end, using modularization and microservices to disassemble the visualization requirements of the entire redis.

7. A method for visualizing cached data, implemented based on the cached data visualization system according to any one of claims 1 to 6, characterized in that, The following steps are involved: Connect to the redis function menu through the command line or front-end page; Global overview, differentiate between role types, and visualize the user side and management side separately.

8. A method for visualizing cached data according to claim 7, characterized in that, The visualization on the user side includes: Users can import other modules to view the list of existing redis clusters; After selecting an instance, enter the instance details page to view the DB databases supported by the current instance and switch them freely. The cluster mode is set with 1 DB database, and the master-slave mode is set with 16 DB databases. After selecting the DB database, users can perform online addition, deletion, modification, and query of data. When querying data, different display formats are supported.

9. A method for visualizing cached data according to claim 7, characterized in that, The visualization on the management side includes: Daily maintenance by redis administrators and authorization of user permissions for redis; Support for redis hot update configuration to update server-side parameters without restarting the service; Support for data migration, setting advanced configuration items to migrate the redis cluster to other clusters; Support for viewing audit logs to record every user operation.

Images