Triggered management system based on dual-control devices

Abstract

The application relates to a trigger type management system based on double control equipment and belongs to the technical field of data storage. The system is logically divided into an interface layer module, a data layer module and a processing layer module, system state data is incrementally updated when a request is received, an interface for full-amount data synchronization is provided, and a complete double control system management scheme is realized. In the application, one controller obtains and integrates the data of another controller through RPC, so that the synchronization of the data is guaranteed; data is directly obtained from a database through a read operation, all peripherals do not need to be traversed, and read operation delay is reduced; a write operation triggers the update of related data, the database does not need to be rewritten in full amount, and write operation delay is reduced.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data storage, and particularly relates to a trigger type management system based on a double-control device. BACKGROUND

[0002] In the field of data storage, especially in the field of centralized storage and disk array, double controllers are a common system redundancy scheme. Double controllers can ensure efficient and stable operation of the entire storage device. Two controllers can provide double read-write bandwidth to the outside at the same time, and can be set in master-slave mode for critical services. Even if the master controller is disabled, the slave controller will automatically take over and continue to provide services, without affecting the entire system's external business. However, double controllers need to display a unified system state to the outside, and need to ensure that the system state of the two controllers is consistent when operating. The traditional solution is to poll the data of the synchronization controller or re-read all state data from the system bottom module. When the number of system-mounted devices increases, the traditional double-control management system will have problems such as double-control data asynchronization, operation lag, and increased system state information update delay. Therefore, designing an efficient double-control system management scheme is crucial to improving the scalability of double-control devices. SUMMARY

[0003] (I) Technical problem to be solved

[0004] The technical problem to be solved by the application is to design a trigger type management system to solve the problems of double-control management data asynchronization and high update delay.

[0005] (II) Technical scheme

[0006] In order to solve the above technical problems, the application provides a design method of a trigger type management system based on a double-control device, which is designed to include:

[0007] An interface layer module for providing calling methods to the outside and distributing request messages to the inside. When the controller is the master controller, the request is directly issued to the data layer module, otherwise the request is forwarded to the opposite controller.

[0008] A data layer module for obtaining and storing system state information from the controller and the opposite controller.

[0009] A processing layer module for providing real-time system state information for the data layer module and executing system operations.

[0010] Preferably, the interface layer module is implemented by FastAPI, and the interface layer module provides four operation methods GET, POST, PUT and DELETE to the outside through the HTTP protocol, which are respectively used for obtaining system information, adding system devices, modifying system information and deleting system devices.

[0011] In the system, the interface layer module maintains a daemon for monitoring the state of the current controller, the controller started first is the master controller, the two controllers are interconnected using the on-board communication interface, the slave controller automatically switches its state and becomes the new master controller after losing the connection with the master controller for a certain time;

[0012] The interface layer module uses an interceptor to set a unified request processing function, the interceptor distributes the request according to the state of the current controller, if the current controller is the master controller, the request is passed down to the data layer module, which is processed by the master controller; if it is the slave controller, the request is forwarded to the data layer module of the opposite controller; no matter which controller processes the request, the result will be returned to the interceptor, which returns uniformly.

[0013] Preferably, the data layer module manages the system state information based on the SQLite database, that is, the state information of the peripherals mounted by the two controllers, the data layer module not only stores the device name and manufacturer name information of the peripherals, but also stores the device state information of the peripherals of the two controllers in the dc1 and dc2 fields; each peripheral mounted in the two controllers provides a function to obtain its own state; the data layer module obtains the state of the current controller by calling the function of the current controller, and obtains the state of the slave controller by calling the function of the opposite controller through RPC, if the opposite controller is lost, the state of the opposite controller is stored as offline, therefore the data layer module always stores the latest state information of the two controllers.

[0014] Preferably, the processing layer module is composed of various function libraries that directly interact with the two controllers, the BlockDev library is used to obtain block device information, the sysfs of the two controllers is used to obtain state information, and the command line tool is used to directly operate the peripheral storage device, although the two controllers mount the same set of peripherals, the operation in a single controller cannot be synchronized to the other controller in time, therefore the processing layer module executes a synchronous refresh operation in the opposite controller through an RPC command after executing the operation in the current controller.

[0015] The application also provides a trigger type management system based on the dual control device designed by the method.

[0016] The application also provides a method for reading system state information by using the system, which comprises the following steps: the read request is first distributed by the interface layer module, after determining the specific request execution controller, the formatted data is directly read from the data layer module of the controller; the data layer module processes the data after paging, sorting and filtering according to the parameters of the request; the data layer module maintains the real-time state information of the system, and directly reads the information from the database.

[0017] The application also provides a method for realizing system operation and data updating by using the system, comprising the following steps: for the system operation of a new RAID, the operation request is distributed by the interface layer module and the operation is guided by the main controller; after receiving the request, the controller to be operated is found according to the index of the data layer module; then the actual operation is performed by calling the corresponding device interaction function library of the processing layer module; finally, the new system state is written back to the data layer module.

[0018] Preferably, the application further comprises the following steps: the operation request distributed by the interface layer module is an event source of the trigger management system, only new requests can trigger the state updating of the data layer module by the processing layer module, and the state updating only allows to modify the data related to the operation request, and does not read all the data for full updating.

[0019] The application also provides a method for realizing data initialization and full synchronization by using the system, comprising the following steps: in the controller starting, controller switching and user manual data synchronization scenarios, the full synchronization function of the processing layer module is called to update the overall state information of the data layer module, the processing layer module first calls the device interaction function library to obtain the real-time state information of all the peripherals of the system, then all the data of the database table of the data layer module is deleted, and finally the new data is inserted into the database table to complete the full synchronization operation.

[0020] Preferably, if there is no old database table during data initialization, the step of deleting the table is skipped.

[0021] (Three) beneficial effects

[0022] The trigger management system based on the double control device designed by the application is logically divided into an interface layer module, a data layer module and a processing layer module, the system state data is incrementally updated when receiving the request, and the interface of data full synchronization is provided, and a complete double control system management scheme is realized. In the application, one controller can obtain and integrate the data of another controller through RPC, so that the synchronization of the data is guaranteed; the data is directly obtained from the database without traversing all the peripherals, so that the read operation delay is reduced; the related data is updated in a trigger mode without rewriting the database in full, so that the write operation delay is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 The figure is a double control device management system architecture of the application;

[0024] Figure 2 The figure is an interface layer module design of the application;

[0025] Figure 3 The figure is a data layer module ER diagram of the application;

[0026] Figure 4This is a flowchart of the trigger-based operation of the present invention. Detailed Implementation

[0027] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0028] This invention designs a trigger-based management system based on dual-controller devices. This system is applied to dual-controller devices and, after optimization and improvement, can be expanded to multiple-controller devices. For example... Figure 1 As shown, the system is divided into an interface layer module, a data layer module, and a processing layer module. The interface layer module is responsible for providing external calling methods and distributing request messages internally. When this controller is the master controller, requests are directly sent to the data layer module; otherwise, the requests are forwarded to the peer controller. The data layer module is responsible for obtaining and storing system information from both this controller and the peer controller. The processing layer module is responsible for providing real-time system status information to the data layer module and executing system operations. In this system, data layer module information is only partially updated upon receiving a request, ensuring data synchronization while reducing system operation latency.

[0029] The interface layer module is designed as follows:

[0030] The system interface layer module is implemented using FastAPI, such as... Figure 2 As shown, the interface layer module provides four operation methods to the outside world via the HTTP protocol: GET, POST, PUT, and DELETE, which are used to obtain system information, add system devices, modify system information, and delete system devices, respectively.

[0031] Internally, the interface layer module maintains a daemon process to monitor the current controller's status. The controller's status is controlled by other modules in the system. The controller that starts first becomes the master controller. The two controllers are interconnected using an onboard communication interface. If a slave controller loses connection to the master controller for a certain period of time, it automatically switches its status and becomes the new master controller.

[0032] The system interface layer module uses interceptors to set up unified request handling functions. The interceptors distribute requests based on the current controller state. If the current controller is the master controller, the request is passed down to the data layer module for processing by this system (master controller); if it is a slave controller, the request is forwarded to the data layer module of the peer controller. Regardless of which controller processes the request, the result is returned to the interceptor for unified return.

[0033] The data layer module is designed as follows:

[0034] The data layer module manages system state information (system state information in the present application refers to state information of external devices (peripherals) mounted by the controller, including power supply, fan state, disk, network card state; it can also include system virtual device state information created based on these peripherals, such as RAID created based on a disk, iSCSI state allocated based on a network card) based on an SQLite database, as shown in FIG. 7. Figure 3 As shown in FIG. 7, the data layer module not only needs to store basic information such as device name and manufacturer name of the peripheral, but also stores device state information of the peripheral of the two controllers in the dcl and dc2 fields. Each peripheral mounted in the dual controller provides a function to obtain its own state, such as whether the disk is online or whether the RAID is degraded. The data layer module will call the function of the local controller to obtain the local controller state, and call the function of the peer controller through RPC to obtain the slave controller state. If the peer controller is lost, the state of the peer controller is stored as offline, so the data layer module always stores the latest dual controller state information.

[0035] The processing layer module is designed as follows:

[0036] The processing layer module is composed of various function libraries that directly interact with the dual controller. BlockDev library is used to obtain block device information, sysfs of the dual controller is used to obtain state information, and command line tools such as mdadm and lvm are used to directly operate peripheral storage devices. Although the dual controller mounts the same set of external devices, the operation in a single controller (system) cannot be synchronized to the other controller in time, so the processing layer module executes a synchronous refresh operation in the peer controller through an RPC command after executing the operation in the local controller. For example, the RAID is created in the local controller, and the RAID is activated in the peer controller.

[0037] The design of the processing layer module not only provides real-time system state information support for the data layer module, but also ensures that the operation issued by the interface layer module is executed atomically in the two controllers, that is, either the operation is successful and the system is stable to the new state, or the operation fails and the system rolls back to the original state.

[0038] System state information reading flow:

[0039] The read request is first distributed by the interface layer module, and after determining the specific request execution controller, the formatted data is directly read from the data layer module of the controller. The data layer module processes the data after paging, sorting, filtering, etc. according to the request parameters, and then returns the formatted data. The data layer module maintains real-time state information of the system to ensure that the latest data can be read, and direct reading from the database also ensures low latency operation.

[0040] System operation and data update flow:

[0041] For new RAID system operation, such as Figure 4 As shown, the operation is also distributed by the interface layer module and dominated by the main controller. After receiving the request, the controller to be operated is first found according to the data layer module index; then the actual operation is performed by the processing layer module calling the corresponding device interaction function library; finally, the new system state is written back to the data layer module.

[0042] The operation request issued by the interface layer module is the event source of the trigger management system. Only new requests can trigger the processing layer module to update the state of the data layer module, and the state update only allows modification of the data related to the operation request, without reading all the data for full update.

[0043] Data initialization and full synchronization process:

[0044] In the scenarios of controller startup, controller switching, and user manual data synchronization, the full synchronization function of the processing layer module needs to be called to update the overall state information of the data layer module. The processing layer module first calls the device interaction function library to obtain the real-time state information of all system peripherals, then deletes all data in the data layer module database table, and finally inserts new data into the database table to complete the full synchronization operation. When initializing data, there is no old database table, so the delete table step is skipped.

[0045] The above only describes the preferred embodiments of the present application. It should be noted that for those skilled in the art, without departing from the technical principles of the present application, a number of improvements and modifications can be made, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. A design method of a trigger-based management system based on a dual-control device, characterized by, The trigger type management system is designed to include: An interface layer module for providing calling methods externally and distributing request messages internally, when the controller is a master controller, the request is directly issued to the data layer module, otherwise the request is forwarded to the opposite controller; A data layer module for obtaining and storing system state information from the controller and the opposite controller; A processing layer module for providing real-time system state information for the data layer module and executing system operations; The interface layer module is implemented by FastAPI, which provides GET, POST, PUT, and DELETE operation methods externally through HTTP protocol, respectively for obtaining system information, adding system devices, modifying system information, and deleting system devices; Internally, the interface layer module maintains a daemon process for monitoring the state of the current controller, the controller started first is the master controller, the two controllers are interconnected using the on-board communication interface, and the slave controller automatically switches its state after losing connection with the master controller for a certain time and becomes the new master controller; The interface layer module uses an interceptor to set a unified request processing function, the interceptor distributes requests based on the state of the current controller, if the current controller is the master controller, the request is passed down to the data layer module, which is processed by the master controller; if it is a slave controller, the request is forwarded to the data layer module of the opposite controller; no matter which controller processes the request, the result will be returned to the interceptor, which will be returned uniformly; The data layer module manages system state information based on SQLite database, i.e. the state information of peripherals mounted by the dual controller, the data layer module not only stores device name and manufacturer name information, but also stores device state information of the two controllers in dc1 and dc2 fields; each peripheral mounted in the dual controller provides a function to obtain its own state; the data layer module obtains the state of the current controller by calling the function of the current controller, and obtains the state of the slave controller by calling the function of the opposite controller through RPC, if the opposite controller is lost, the state of the opposite controller is stored as offline, so the data layer module always stores the latest dual controller state information.

2. The method of claim 1, wherein, The processing layer module is composed of various function libraries that directly interact with the dual controller, which obtains block device information through BlockDev library, obtains state information through sysfs of the dual controller, and directly operates peripheral storage devices through command line tools, although the dual controller mounts the same set of peripherals, the operation in a single controller cannot be synchronized to the other controller in time, therefore the processing layer module executes a synchronous refresh operation in the opposite controller through RPC command after executing the operation in the current controller.

3. A trigger type management system based on dual control devices designed by the method of claim 2.

4. A method for reading system status information using the system of claim 3, wherein, Including the following steps: The read request is first distributed by the interface layer module, after determining the specific request execution controller, the formatted data is returned after paging, sorting, and filtering processing according to the parameters of the request; The data layer module returns the formatted data after paging, sorting, and filtering processing according to the parameters of the request; The data layer module maintains the system real-time state information and reads information directly from the database.

5. A method for implementing system operation and data update using the system as claimed in claim 3, characterized in that, The method comprises the following steps: For the system operation of a new RAID, the interface layer module distributes the operation, and the main controller leads the operation; after receiving the request, the controller to be operated is found according to the data layer module index; then the processing layer module calls the corresponding device interaction function library to perform actual operation; finally, the new system state is written back to the data layer module.

6. The method of claim 5, wherein, The method further comprises the following steps: The operation request distributed by the interface layer module is an event source of the trigger type management system, only a new request can trigger the processing layer module to update the state of the data layer module, and the state update only allows modification of the data related to the operation request, and does not read all data for full update.

7. A method for implementing data initialization and full volume synchronization using the system of claim 3, characterized in that, The method comprises the following steps: in In the controller startup, controller switching and user manual data synchronization scenarios, the full-amount synchronization function of the processing layer module is called to update the overall state information of the data layer module, the processing layer module first calls the device interaction function library to obtain the real-time state information of all peripherals of the system, then all data in the database table of the data layer module is deleted, and finally the new data is inserted into the database table, thereby completing the full-amount synchronization operation.

8. The method of claim 7, wherein, If there is no old database table during data initialization, the step of deleting the table is skipped.

Images