Data reading and writing method and device, electronic equipment and storage medium

Abstract

The application provides a data read-write method and device, electronic equipment and a storage medium, and relates to the technical field of data processing. The method comprises the following steps: monitoring a data processing request of a server end, the data processing request being used for requesting configuration data of the server end; in response to the data processing request, summarizing configuration data generated by the server end; writing the configuration data of the server end into a distributed storage cluster through a data processing interface pre-packaged by the distributed storage cluster; and in response to a read configuration data request of the server end corresponding to a back-end storage, synchronizing the configuration data to the back-end storage through the data processing interface. The application realizes reliable storage of global configuration data of the server end through the distributed storage cluster, prevents single-point failure, ensures consistency of the global configuration data, and further improves the data synchronization effect of an iSCSI server.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data reading and writing method, apparatus, electronic device, and storage medium. Background Technology

[0002] iSCSI (Internet Small Computer System Interface) combines the existing iSCSI interface with Ethernet technology, connecting the iSCSI server (Target) and client (Initiator) based on the TCP / IP protocol. This allows encapsulated iSCSI data packets to be transmitted over the general Internet, ultimately enabling the iSCSI server to be mapped as a storage space (disk) provided to the connected and authenticated client. Because iSCSI can send the SCSI protocol, which was originally only used locally, through the TCP / IP network, the connection distance can be extended infinitely geographically, and the number of connected servers is unlimited, enabling online expansion and even dynamic deployment.

[0003] Currently, configuration data in the iSCSI server (Target) is distributed and stored. Volume mapping information in the configuration data is stored in local configuration files. Therefore, if configuration data needs to be synchronized, it is obtained by reading and writing local configuration files. Each synchronization node needs to perform this operation once, resulting in high command execution time. If a node fails, it may cause inconsistencies with the configuration data of other nodes after recovery, affecting the data synchronization effect of the iSCSI server. Summary of the Invention

[0004] In view of this, the present invention aims to provide a data reading and writing method, apparatus, electronic device and storage medium to solve the problem of low efficiency in obtaining configuration data by reading and writing local configuration files, and to achieve efficient and reliable data reading and writing.

[0005] According to a first aspect of the present invention, a data read / write method is provided, applied to a distributed storage cluster, the method comprising:

[0006] Monitor server-side data processing requests; the data processing requests are used to request configuration data stored on the server side.

[0007] In response to the data processing request, the configuration data generated by the server is summarized;

[0008] The configuration data on the server side is written into the distributed storage cluster through a pre-encapsulated data processing interface of the distributed storage cluster;

[0009] In response to a request to read configuration data from the corresponding backend storage on the server side, the configuration data is synchronized to the backend storage through the data processing interface.

[0010] Optionally, the monitoring server-side data processing request; the data processing request is used to request configuration data stored on the server-side, including:

[0011] A preset monitoring thread is used to monitor data processing requests sent by the server.

[0012] The data processing request is parsed to determine that the server requests the configuration data of the distributed storage cluster storage server.

[0013] Optionally, the step of summarizing the configuration data generated by the server in response to the data processing request includes:

[0014] In response to the data processing request, the global configuration of the server is traversed to determine the volume mapping configuration information, path number information and authentication information generated by the server.

[0015] The volume mapping configuration information, path number information, and authentication information are cached in a queue to obtain summarized configuration data.

[0016] Optionally, writing the configuration data from the server to the distributed storage cluster via a pre-encapsulated data processing interface includes:

[0017] The pre-defined interfaces in the distributed storage cluster are pre-encapsulated into various types of data processing interfaces; wherein, the data processing interfaces include data writing interfaces and data modification interfaces;

[0018] In response to the write configuration data request from the server, the data write interface corresponding to the write configuration data request in the distributed storage cluster is invoked;

[0019] The configuration data from the server is written to the preset storage area of ​​the distributed storage cluster through the data writing interface.

[0020] In response to the server-side request to modify configuration data, a modification operation on the configuration data of the preset storage area is determined;

[0021] Call the data modification interface corresponding to the configuration data modification request in the distributed storage cluster;

[0022] The modification operation is performed through the data modification interface to update the configuration information of the preset storage area.

[0023] Optionally, the data processing interface includes a data reading interface, and the step of responding to a read configuration data request from the server-side corresponding backend storage and synchronizing the configuration data to the backend storage through the data processing interface includes:

[0024] In response to the read configuration data request from the backend storage on the server side, the data read interface corresponding to the read configuration data request in the distributed storage cluster is called;

[0025] The configuration data of the preset storage area of ​​the distributed storage cluster is synchronized to the backend storage through the data reading interface.

[0026] Optionally, before the data processing request from the monitoring server, the following steps are also included:

[0027] A distributed storage cluster is pre-deployed on the server side, and a connection is created between the distributed storage cluster and the server side; wherein, the distributed cluster includes a master node and a slave node for reading and writing data.

[0028] Optionally, after responding to a read configuration data request from the server-side corresponding backend storage and synchronizing the configuration data to the backend storage via the data processing interface, the process further includes:

[0029] Monitor the status of the master nodes of the distributed storage cluster; wherein, the distributed storage cluster includes a cluster alarm interface;

[0030] If an abnormality is detected in the status of the master node, a target master node is selected from the slave nodes, and the configuration data is backed up to the target master node;

[0031] The cluster alarm interface is used to send an error message to the server.

[0032] According to a second aspect of the present invention, a data read / write apparatus is provided, applied to a distributed storage cluster, the apparatus comprising:

[0033] The monitoring request module is used to monitor data processing requests from the server; the data processing requests are used to request configuration data stored on the server.

[0034] The data aggregation module is used to aggregate the configuration data generated by the server in response to the data processing request.

[0035] The data writing module is used to write the configuration data on the server side into the distributed storage cluster through a data processing interface pre-encapsulated by the distributed storage cluster.

[0036] The data reading module is used to respond to the server-side request to read configuration data from the corresponding backend storage and synchronize the configuration data to the backend storage through the data processing interface.

[0037] According to another aspect of the present invention, an electronic device is also provided, comprising:

[0038] processor;

[0039] Memory used to store the processor's executable instructions;

[0040] The processor is configured to execute the instructions to implement the data read / write method described above.

[0041] According to another aspect of the present invention, a readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the data read / write method described above.

[0042] The data read / write method provided in this invention monitors data processing requests from the server, and in response to these requests, aggregates configuration data generated by the server. This server-side configuration data is then written to the distributed storage cluster via a pre-encapsulated data processing interface. In response to read configuration data requests from the corresponding backend storage on the server, the configuration data is synchronized to the backend storage via the data processing interface. This invention utilizes a distributed storage cluster to reliably store global configuration data on the server, preventing single points of failure. The distributed storage cluster can adapt to the growth of server-side data volume. By promptly responding to read / write requests, it ensures that the backend storage and server can quickly synchronize configuration data, achieving efficient, reliable, and flexible processing of server-side configuration data, guaranteeing global configuration data consistency, and further improving the data synchronization effect of the iSCSI server.

[0043] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0044] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0045] Figure 1 This is a flowchart of the steps of a data reading and writing method provided in an embodiment of the present invention;

[0046] Figure 2 yes Figure 1 A flowchart of step 101 in the data read / write method provided in this embodiment of the invention;

[0047] Figure 3 yes Figure 1 A flowchart of step 102 in the data read / write method provided in this embodiment of the invention;

[0048] Figure 4 yes Figure 1 A flowchart of step 103 in the data read / write method provided in this embodiment of the invention;

[0049] Figure 5 yes Figure 1 A flowchart of step 104 in the data read / write method provided in this embodiment of the invention;

[0050] Figure 6 This is a flowchart of another data read / write method provided in an embodiment of the present invention;

[0051] Figure 7 This is a schematic diagram of a data read / write method provided in an embodiment of the present invention;

[0052] Figure 8 This is a schematic diagram of the structure of a data read / write device provided in an embodiment of the present invention;

[0053] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details are presented in the various embodiments of the present invention to facilitate a better understanding of this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for ease of description and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined with and referenced by each other without contradiction.

[0055] Reference Figure 1 The diagram illustrates a flowchart of the data read / write method provided in an embodiment of the present invention, applied to a distributed storage cluster. The method may include:

[0056] Step 101: Monitor the data processing requests on the server side; the data processing requests are used to request configuration data stored on the server side.

[0057] In this embodiment of the invention, the iSCSI protocol mainly includes two parts: an iSCSI initiator and an iSCSI target. The initiator, or initiating end, is deployed on the client side, and the target, or destination end, is deployed on the storage server side. The iSCSI protocol is the process of mapping iSCSI remote procedure calls to the IP protocol. (Refer to...) Figure 7 This illustration shows a scenario diagram of a data read / write method provided by an embodiment of the present invention. In this embodiment, a distributed storage cluster is built, and the global configuration information of the iSCSI target is stored in the distributed storage cluster in a key / value manner. When the iSCSI server is working, the application data read and write is directly written to the backend storage. Configuration data, such as volume mapping information, path information, and chapter information, are written to the distributed storage cluster. If the backend storage needs to obtain the configuration data of the iSCSI target, it can read it from the distributed storage cluster. By storing the global configuration information of the iSCSI target through the distributed storage cluster, the low efficiency problem of reading and writing data from local files in the traditional method is avoided. It can support a larger amount of configuration data and enable the backend storage to share global configuration information with the iSCSI server.

[0058] It should be noted that the distributed storage cluster in this embodiment uses etcd (etc Distribute). etcd is a distributed consistent key-value storage system used for shared configuration and service discovery. It is a highly available distributed key-value storage database. etcd internally uses the Raft protocol as the consistency algorithm. The Raft protocol algorithm is a distributed consistency algorithm used to ensure data consistency in a distributed system.

[0059] Specifically, when the server starts, the distributed storage cluster initializes a data processing request monitoring mechanism. The monitoring mechanism can adopt polling, event-driven, or other methods. The monitoring mechanism continuously listens for data processing requests on the server. Once a request is received, it is processed immediately. The received data processing request can be parsed to extract information such as the request type, content, and target storage location. The request is then classified into different processing tasks, such as configuration data storage requests and configuration data read requests.

[0060] Step 102: In response to the data processing request, summarize the configuration data generated by the server.

[0061] In this embodiment of the invention, when the monitoring mechanism detects a data processing request, it immediately responds to the request and initiates a configuration data aggregation process. This process collects generated configuration data from various modules or services on the server side, and aggregates the collected configuration data to ensure data integrity and consistency. The aggregation process may include operations such as data merging, deduplication, and formatting. The aggregated configuration data is then stored in a temporary storage area, such as a memory cache or a local file system, for subsequent processing.

[0062] It should be noted that the specific global configuration information related to the iSCSI target includes the following: Volume mapping configuration information: Key: pool_name (storage pool name) and target_name (iSCSI server target name), Value: version (initialized to 0), lun_id (volume name), target_ip_list (list of IPs of the bound host), target_initor_list (list of bound initiators); Path number information: Key: path_num (number of paths), value: max_path_num (maximum number of paths), min_path_num (minimum number of paths); Authentication information: Key: chap_user (CHAP username), Value: including incoming / outcoming (authentication direction), version (initialized to 0), password (encrypted password), initor_list (list of bound initiators).

[0063] Step 103: Write the configuration data from the server to the distributed storage cluster through the pre-encapsulated data processing interface of the distributed storage cluster.

[0064] In this embodiment of the invention, configuration data from the server is written to the distributed storage cluster through a pre-encapsulated data processing interface. The pre-encapsulated interfaces in the distributed storage cluster are divided into various types of data processing interfaces, including a data write interface and a data modification interface. In response to a server-side request to write configuration data, the distributed storage cluster calls the corresponding data write interface to write the server-side configuration data to a pre-defined storage area within the distributed storage cluster. In response to a server-side request to modify configuration data, a modification operation is determined for the configuration data in the pre-defined storage area. The corresponding data modification interface is then called to perform the modification operation and update the configuration information in the pre-defined storage area.

[0065] Step 104: In response to the server-side request to read configuration data from the corresponding backend storage, synchronize the configuration data to the backend storage through the data processing interface.

[0066] In this embodiment of the invention, when the iSCSI server is working, application data is directly written to the back-end storage. The back-end storage corresponding to the server includes physical storage devices such as disk arrays (RAID) or virtual storage devices such as virtual disks, etc., without specific limitations. The application data of the iSCSI server can be directly stored in the back-end device. When the back-end storage needs global configuration data when running application data, it requests to read the configuration data from the distributed storage cluster. The distributed storage cluster responds to the request to read the configuration data from the back-end storage corresponding to the server and synchronizes the configuration data to the back-end storage through the data processing interface.

[0067] The data read / write method provided in this invention monitors data processing requests from the server, and in response to these requests, aggregates configuration data generated by the server. This server-side configuration data is then written to the distributed storage cluster via a pre-encapsulated data processing interface. In response to read configuration data requests from the corresponding backend storage on the server, the configuration data is synchronized to the backend storage via the data processing interface. This invention utilizes a distributed storage cluster to reliably store global configuration data on the server, preventing single points of failure. The distributed storage cluster can adapt to the growth of server-side data volume. By promptly responding to read / write requests, it ensures that the backend storage and server can quickly synchronize configuration data, achieving efficient, reliable, and flexible processing of server-side configuration data, guaranteeing global configuration data consistency, and further improving the data synchronization effect of the iSCSI server.

[0068] Furthermore, refer to Figure 2 , showed Figure 1 A flowchart of step 101 in a data read / write method is provided. This method is basically the same as the data read / write method provided in the first embodiment of the present invention. Step 101 may include:

[0069] Step 201: Use a preset monitoring thread to monitor the data processing requests sent by the server.

[0070] Step 202: Parse the data processing request to determine the configuration data requested by the server from the distributed storage cluster storage server.

[0071] It should be noted that, in this embodiment of the invention, when the server starts, the distributed storage cluster initializes one or more monitoring threads. The monitoring threads are responsible for listening to data processing requests from the server. Specifically, a specific listening port can be assigned to the monitoring thread to ensure that data processing requests from the server can be sent through the port. The monitoring thread continuously listens to the specified port, and once a data processing request is received, it is immediately cached in memory or temporary storage for subsequent processing.

[0072] In this embodiment, cached data processing requests can be initially filtered and classified to identify requests that need to be processed by the distributed storage cluster. Data processing requests are extracted from the cache, and a predefined parser is used to parse the request content to extract key information, such as request type, data type, and storage requirements. The parsed requests are then verified to ensure their legality and integrity. Based on the parsed request information, the configuration data type and storage location that the server needs to store are determined so that the configuration data can be stored in the distributed storage cluster.

[0073] This invention, through a preset monitoring thread, can capture data processing requests from the server in real time, ensuring that requests can be processed promptly. By storing configuration data through a distributed storage cluster, it improves data reliability and redundancy and prevents single points of failure.

[0074] Furthermore, refer to Figure 3 , showed Figure 1 A flowchart of step 102 in a data read / write method is provided. This method is basically the same as the data read / write method provided in the first embodiment of the present invention. Step 102 may include:

[0075] Step 301: In response to the data processing request, traverse the global configuration on the server side to determine the volume mapping configuration information, path number information and authentication information generated by the server side.

[0076] Step 302: The volume mapping configuration information, path number information, and authentication information are cached in a queue to obtain the summarized configuration data.

[0077] It should be noted that, in this embodiment of the invention, the distributed storage cluster responds to the data processing request from the server by traversing the global configuration of the server to determine the volume mapping configuration information, path number information, and authentication information generated by the server. Specifically, when the monitoring thread captures the data processing request, it immediately responds to the request and starts the configuration traversal process to traverse the global configuration of the server and obtain all relevant configuration information. In this embodiment, the traversal can be performed recursively or iteratively to ensure that no configuration item is missed.

[0078] Specifically, during the traversal, configuration information related to volume mapping is identified and extracted, including the volume name, mapping path, access permissions, etc.; the traversal continues, extracting configuration information related to the number of paths, including the number of paths, the hierarchical structure of paths, the access frequency of paths, etc.; authentication information is extracted; configuration information related to authentication is extracted, including user authentication, access control, access tokens, etc. The volume mapping configuration information, path count information, and authentication information are queued and cached to obtain the summarized configuration data.

[0079] In this embodiment, one or more queue caches are initialized on the server side to temporarily store the extracted configuration information. The queue cache can manage data in a first-in, first-out (FIFO) manner, sequentially enqueuing the extracted volume mapping configuration information, path count information, and authentication information. Each configuration information item is treated as an independent queue element, and the queue is managed to ensure that the queue capacity and performance meet the requirements. After all configuration information has been successfully enqueued, the data in the queue is summarized. The summarization process may include operations such as data merging, deduplication, and formatting to ensure that the final configuration data is complete and consistent, so that the summarized configuration data can be stored in the specified storage location of the distributed storage cluster.

[0080] This invention embodiment ensures that all relevant configuration information is extracted through global configuration traversal, enabling it to respond quickly and accurately to data processing requests and generate summarized configuration data. Queue caching ensures that configuration information is not lost during processing, improving the efficiency of configuration information extraction and reducing processing time.

[0081] Furthermore, refer to Figure 4 , showed Figure 1 A flowchart of step 103 in a data read / write method is provided. This method is basically the same as the data read / write method provided in the first embodiment of the present invention. Step 103 may include:

[0082] Step 401: Pre-encapsulate the preset interfaces in the distributed storage cluster into various types of data processing interfaces; wherein, the data processing interfaces include data writing interfaces and data modification interfaces;

[0083] Step 402: In response to the write configuration data request from the server, call the data write interface corresponding to the write configuration data request in the distributed storage cluster;

[0084] Step 403: Write the configuration data from the server to the preset storage area of ​​the distributed storage cluster through the data writing interface;

[0085] Step 404: In response to the server's request to modify configuration data, determine the modification operation of configuration data in the preset storage area;

[0086] Step 405: Call the data modification interface corresponding to the configuration data modification request in the distributed storage cluster;

[0087] Step 406: Perform modification operations through the data modification interface to update the configuration information of the preset storage area.

[0088] In this embodiment of the invention, pre-defined interfaces in the distributed storage cluster are encapsulated into various types of data processing interfaces. These data processing interfaces include data write interfaces and data modification interfaces. Specifically, in the distributed storage cluster, various types of data processing interfaces are pre-defined. These interfaces can be categorized according to storage requirements, including data write interfaces, data modification interfaces, and data read interfaces. For example, in this embodiment, the encapsulated data processing interfaces undergo comprehensive testing to ensure their stability and performance under different scenarios. The interfaces encapsulating storage configuration data include data write interfaces, data read interfaces, and data modification interfaces. Specifically, the data write interface is `wirite_iscsi_conf_data(key, value)`, the data read interface is `read_iscsi_conf_data(key, value)`, and the data modification interface is `modify_iscsi_conf_data(key, value)`.

[0089] Specifically, in response to a write configuration data request from the server, the corresponding data write interface in the distributed storage cluster is invoked. Upon receiving the write configuration data request, the distributed storage cluster immediately responds to the request and initiates the data write process. Based on the request content, it invokes the pre-encapsulated data write interface in the distributed storage cluster, executes the data write interface, and writes the configuration data into the distributed storage cluster. The configuration data from the server is written to the preset storage area of ​​the distributed storage cluster through the data write interface. Based on the type of configuration data and access requirements, a suitable preset storage area is selected. The storage area can be a node in the distributed storage cluster or a specific storage partition.

[0090] Specifically, in this embodiment, after receiving a request to modify configuration data, the distributed storage cluster immediately responds to the request and initiates a data modification process. Based on the request content, it determines the necessary modification operations. These operations include data updates, deletions, and appends. The cluster locates the configuration data to be modified in a preset storage area to ensure the accuracy of the modification operation. Depending on the modification operation type, it calls a pre-encapsulated data modification interface within the distributed storage cluster, executes the interface, and modifies the configuration data in the preset storage area. Through the data modification interface, it executes the determined modification operation, updating the configuration information in the preset storage area. After the modification operation is completed, the modified configuration data is verified to ensure the successful execution of the modification operation.

[0091] This invention, through pre-encapsulating a data processing interface, responds to write and modify requests and executes data write and modify operations, achieving efficient, reliable, and flexible processing of server-side configuration data. By using a pre-defined storage area in a distributed storage cluster, it improves data reliability and redundancy, prevents single points of failure, and allows the distributed storage cluster to dynamically expand storage capacity and processing capabilities as needed to adapt to the growth of server-side data volume. Through reasonable modification operation execution and data verification strategies, it ensures that configuration data can be updated quickly and efficiently.

[0092] Furthermore, refer to Figure 5 , showed Figure 1 A flowchart of step 104 in a data read / write method is provided. This method is basically the same as the data read / write method provided in the first embodiment of the present invention. The data processing interface includes a data reading interface, and step 104 may include:

[0093] Step 501: In response to the read configuration data request from the corresponding backend storage on the server side, call the data read interface corresponding to the read configuration data request in the distributed storage cluster.

[0094] Step 502: Synchronize the configuration data of the preset storage area of ​​the distributed storage cluster to the backend storage through the data reading interface.

[0095] It should be noted that, in this embodiment of the invention, the distributed storage cluster responds to the read configuration data request from the corresponding backend storage on the server side. Upon receiving the read configuration data request, it immediately responds to the request, initiates the data read process, calls the data read interface corresponding to the read configuration data request in the distributed storage cluster, executes the data read interface, and reads the configuration data from the distributed storage cluster. Specifically, through the data read interface, the configuration data is read from the preset storage area of ​​the distributed storage cluster, and the read configuration data is synchronized to the backend storage on the server side. The synchronization process can adopt a full synchronization or incremental synchronization method, and an appropriate synchronization strategy can be selected according to actual needs. After the synchronization operation is completed, the synchronized configuration data is verified to ensure the successful execution of the synchronization operation.

[0096] In this embodiment of the invention, the back-end storage of the iSCSI server refers to the physical or virtual storage devices that actually store the data. These devices communicate with the client through the iSCSI protocol. The back-end storage of the iSCSI server includes physical storage devices such as disk arrays (RAID) or virtual storage devices such as virtual disks, etc., without specific limitations. Application data of the iSCSI server can be directly stored in the back-end devices.

[0097] This invention improves data reliability and redundancy by using a pre-defined storage area in a distributed storage cluster, preventing single points of failure. The distributed storage cluster can dynamically expand its storage capacity and processing capabilities according to demand, adapting to the growth of server-side data volume. Through a reasonable synchronization strategy, it ensures that configuration data can be quickly and efficiently synchronized to the backend storage, enabling the backend storage and the iSCSI server target to share global configuration information.

[0098] Reference Figure 6 The diagram illustrates a flowchart of another data read / write method provided by an embodiment of the present invention. This method is essentially the same as the data read / write method provided by the first embodiment of the present invention, except that the method may further include:

[0099] Step 105: Deploy a distributed storage cluster on the server side in advance and create a connection between the distributed storage cluster and the server side; wherein, the distributed cluster includes a master node and a slave node for reading and writing data.

[0100] In this embodiment of the invention, a distributed storage cluster is pre-deployed on the server side to build a multi-node etcd cluster. A connection is created between the distributed storage cluster and the server side. The distributed cluster includes master nodes and slave nodes for reading and writing data to ensure data fault tolerance. In this embodiment, the split-brain phenomenon is avoided by deploying an odd number of nodes, such as 4 nodes. If 2 nodes are on different networks from the other 2 nodes, it is equivalent to the cluster being divided into two network regions, which will cause the entire etcd cluster to become unusable. Alternatively, there may be two master ends, meaning that the master nodes in both regions can be used, and the system cannot determine which region to select for configuring data storage. If there is an odd number of nodes, etcd internally follows the majority rule, meaning that as long as the number of normally usable nodes is greater than half of the total number of nodes in the original cluster, the cluster can provide services normally.

[0101] Step 101: Monitor the data processing requests on the server side; the data processing requests are used to request configuration data stored on the server side.

[0102] Step 102: In response to the data processing request, summarize the configuration data generated by the server.

[0103] Step 103: Write the configuration data on the server side into the distributed storage cluster through the data processing interface pre-encapsulated by the distributed storage cluster;

[0104] Step 104: In response to the server-side request to read configuration data from the corresponding backend storage, synchronize the configuration data to the backend storage through the data processing interface.

[0105] Based on the beneficial effects of the first embodiment, this embodiment of the invention pre-deploys a distributed storage cluster on the server side and creates a connection between the distributed storage cluster and the server side. Through the distributed storage cluster, a highly available storage solution is provided to prevent single points of failure. The distributed storage cluster supports dynamic expansion to meet the ever-increasing storage needs of the server-side configuration data.

[0106] Specifically, after responding to a request to read configuration data from the server-side backend storage and synchronizing the configuration data to the backend storage via the data processing interface, the process further includes:

[0107] First, monitor the status of the master node of the distributed storage cluster; wherein, the distributed storage cluster includes a cluster alarm interface;

[0108] Secondly, if an abnormality is detected in the status of the master node, a target master node is selected from the slave nodes, and the configuration data is backed up to the target master node;

[0109] Secondly, the cluster alarm interface is used to send an abnormality alert to the server.

[0110] It should be noted that, in this embodiment of the invention, when the distributed storage cluster starts, a master node status monitoring mechanism is initialized. This monitoring mechanism can employ methods such as heartbeat detection and status polling. The master node periodically sends heartbeat signals to the monitoring mechanism to indicate its normal operation. The monitoring mechanism continuously receives heartbeat signals and records the master node's status. The monitoring mechanism periodically polls the master node's status information, including memory usage and I / O, to ensure the master node's health. A cluster alarm interface is initialized to send an error message to the server when the master node's status is abnormal. If an abnormal master node status is detected, a target master node is selected from the secondary nodes, and the configuration data is backed up to the target master node. The selection process can be based on factors such as the secondary node's load, health status, and historical performance. The configuration data in the master node is backed up to the target master node. The backup process should ensure data integrity and consistency. After the configuration data backup is completed, the target secondary node is switched to the master node to ensure the normal operation of the distributed storage cluster.

[0111] Specifically, the cluster alarm interface is used to send abnormal prompt information to the server. After detecting an abnormal status of the master node and completing the election of the slave node and backup of the configuration data, an abnormal prompt information is generated. The alarm information should include detailed information such as the abnormality type, abnormal time, and handling measures. The cluster alarm interface is called to send the generated abnormal prompt information to the server. The server can record the sending log of the alarm information, including the sending time, recipient, alarm content, etc., to facilitate subsequent troubleshooting.

[0112] This invention ensures high availability of the distributed storage cluster and prevents single points of failure by electing secondary nodes and switching primary nodes. The data backup process ensures data consistency and avoids data loss or corruption. Through the cluster alarm interface, abnormal prompts can be sent to the server in a timely manner to ensure that problems can be responded to and handled quickly, thus achieving high availability and data consistency of the distributed storage cluster.

[0113] Reference Figure 8 The diagram illustrates a data read / write device according to an embodiment of the present invention, applied to a distributed storage cluster. The device includes:

[0114] The monitoring request module 601 is used to monitor data processing requests from the server; the data processing requests are used to request configuration data stored on the server.

[0115] Data aggregation module 602 is used to aggregate the configuration data generated by the server in response to the data processing request;

[0116] The data writing module 603 is used to write the configuration data on the server side into the distributed storage cluster through the data processing interface pre-encapsulated by the distributed storage cluster;

[0117] The data reading module 604 is used to respond to the server-side request to read configuration data from the corresponding backend storage and synchronize the configuration data to the backend storage through the data processing interface.

[0118] Furthermore, the monitoring request module 601 includes:

[0119] The monitoring submodule is used to monitor data processing requests sent by the server using a preset monitoring thread;

[0120] The parsing submodule is used to parse the data processing request and determine the configuration data of the distributed storage cluster storage server requested by the server.

[0121] Furthermore, the data aggregation module 602 includes:

[0122] The traversal submodule is used to traverse the global configuration of the server in response to the data processing request, and determine the volume mapping configuration information, path number information and authentication information generated by the server.

[0123] The caching submodule is used to cache the volume mapping configuration information, path number information, and authentication information in a queue to obtain summarized configuration data.

[0124] Furthermore, the data writing module 603 includes:

[0125] An encapsulation submodule is used to pre-encapsulate preset interfaces in the distributed storage cluster into various types of data processing interfaces; wherein, the data processing interfaces include data writing interfaces and data modification interfaces;

[0126] The first calling submodule is used to respond to the write configuration data request on the server side and call the data write interface corresponding to the write configuration data request in the distributed storage cluster;

[0127] The write submodule is used to write the configuration data on the server side to the preset storage area of ​​the distributed storage cluster through the data write interface;

[0128] The determination submodule is used to respond to the server-side request to modify configuration data and determine the modification operation of configuration data in the preset storage area;

[0129] The second calling submodule is used to call the data modification interface corresponding to the configuration data modification request in the distributed storage cluster;

[0130] The modification submodule is used to perform the modification operation through the data modification interface and update the configuration information of the preset storage area.

[0131] Furthermore, the data processing interface includes a data reading interface, and the data reading module 604 includes:

[0132] The third calling submodule is used to respond to the read configuration data request of the corresponding backend storage on the server side and call the data read interface corresponding to the read configuration data request in the distributed storage cluster;

[0133] The synchronization submodule is used to synchronize the configuration data of the preset storage area of ​​the distributed storage cluster to the backend storage through the data reading interface.

[0134] Furthermore, the device also includes:

[0135] A cluster creation module is used to pre-deploy a distributed storage cluster on the server side and create a connection between the distributed storage cluster and the server side; wherein, the distributed cluster includes a master node and a slave node for reading and writing data.

[0136] Furthermore, the device also includes:

[0137] The monitoring node module is used to monitor the status of the master node of the distributed storage cluster; wherein, the distributed storage cluster includes a cluster alarm interface;

[0138] The data backup module is used to select a target master node from the slave nodes and back up the configuration data to the target master node if an abnormal state of the master node is detected.

[0139] The exception notification module is used to send exception notification information to the server using the cluster alarm interface.

[0140] The data read / write device provided in this embodiment of the invention monitors data processing requests from the server, and in response to these requests, aggregates configuration data generated by the server. It then writes this server-side configuration data into the distributed storage cluster via a pre-encapsulated data processing interface. In response to read configuration data requests from the corresponding backend storage on the server, it synchronizes the configuration data to the backend storage via the data processing interface. This embodiment of the invention achieves reliable storage of global configuration data on the server through a distributed storage cluster, preventing single points of failure. The distributed storage cluster can adapt to the growth of server-side data volume. By promptly responding to read / write requests, it ensures that the backend storage and the server can quickly synchronize configuration data, achieving efficient, reliable, and flexible processing of server-side configuration data, ensuring global configuration data consistency, and further improving the data synchronization effect of the iSCSI server.

[0141] Reference Figure 9 The present invention also provides an electronic device, such as... Figure 9 As shown, it includes a processor 701, a communication interface 702, a memory 703, and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0142] Memory 703 is used to store computer programs;

[0143] When processor 701 executes a program stored in memory 703, it performs the following steps:

[0144] Monitor server-side data processing requests; the data processing requests are used to request configuration data stored on the server side.

[0145] In response to the data processing request, the configuration data generated by the server is summarized;

[0146] The configuration data on the server side is written into the distributed storage cluster through a pre-encapsulated data processing interface of the distributed storage cluster;

[0147] In response to a request to read configuration data from the corresponding backend storage on the server side, the configuration data is synchronized to the backend storage through the data processing interface.

[0148] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0149] The communication interface is used for communication between the aforementioned terminal and other devices.

[0150] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0151] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0152] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the data read / write methods described in the above embodiments.

[0153] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data reading and writing device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0154] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0155] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0156] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A data read / write method, characterized in that, Applied to a distributed storage cluster, wherein the distributed cluster includes a master node and a slave node for reading and writing data, the method includes: Monitor server-side data processing requests; the data processing requests are used to request configuration data stored on the server side. In response to the data processing request, the configuration data generated by the server is aggregated. The configuration data is cached by one or more queues initialized by the server. The extracted volume mapping configuration information, path number information and authentication information are sequentially enqueued. The data in the queue is aggregated and stored in the distributed storage cluster storage location. The configuration data on the server side is written to the distributed storage cluster through a pre-encapsulated data processing interface; wherein, the data processing interface is obtained by pre-encapsulating a preset interface in the distributed storage cluster into multiple types, and the data processing interface includes a data writing interface and a data modification interface; In response to a request to read configuration data from the corresponding backend storage on the server side, the configuration data is synchronized to the backend storage through the data processing interface; Monitor the status of the master nodes of the distributed storage cluster; wherein, the distributed storage cluster includes a cluster alarm interface; If an abnormality is detected in the status of the master node, a target master node is selected from the slave nodes, and the configuration data is backed up to the target master node; The cluster alarm interface is used to send an error message to the server.

2. The method according to claim 1, characterized in that, The data processing request from the monitoring server; The data processing request is used to request configuration data from the storage server, including: A preset monitoring thread is used to monitor data processing requests sent by the server. The data processing request is parsed to determine that the server requests the configuration data of the distributed storage cluster storage server.

3. The method according to claim 1, characterized in that, The step of summarizing the configuration data generated by the server in response to the data processing request includes: In response to the data processing request, the global configuration of the server is traversed to determine the volume mapping configuration information, path number information and authentication information generated by the server. The volume mapping configuration information, path number information, and authentication information are cached in a queue to obtain summarized configuration data.

4. The method according to claim 1, characterized in that, The step of writing the configuration data from the server to the distributed storage cluster through a pre-encapsulated data processing interface includes: The pre-defined interfaces in the distributed storage cluster are pre-encapsulated into various types of data processing interfaces; wherein, the data processing interfaces include data writing interfaces and data modification interfaces; In response to the write configuration data request from the server, the data write interface corresponding to the write configuration data request in the distributed storage cluster is invoked; The configuration data from the server is written to the preset storage area of ​​the distributed storage cluster through the data writing interface. In response to the server-side request to modify configuration data, a modification operation on the configuration data of the preset storage area is determined; Call the data modification interface corresponding to the configuration data modification request in the distributed storage cluster; The modification operation is performed through the data modification interface to update the configuration information of the preset storage area.

5. The method according to claim 1, characterized in that, The data processing interface includes a data reading interface. The step of responding to a server-side request to read configuration data from the corresponding backend storage and synchronizing the configuration data to the backend storage via the data processing interface includes: In response to the read configuration data request from the backend storage on the server side, the data read interface corresponding to the read configuration data request in the distributed storage cluster is called; The configuration data of the preset storage area of ​​the distributed storage cluster is synchronized to the backend storage through the data reading interface.

6. The method according to claim 1, characterized in that, Before the data processing request from the monitoring server, the following is also included: A distributed storage cluster is pre-deployed on the server side, and a connection is created between the distributed storage cluster and the server side.

7. A data read / write device, characterized in that, The device is applied to a distributed storage cluster, wherein the distributed cluster includes a master node and a slave node for reading and writing data, and the device includes: The monitoring request module is used to monitor data processing requests from the server; the data processing requests are used to request configuration data stored on the server. The data aggregation module is used to respond to the data processing request, aggregate the configuration data generated by the server, and the configuration data is cached by one or more queues initialized by the server. The extracted volume mapping configuration information, path number information and authentication information are sequentially enqueued, the data in the queues are aggregated and stored in the distributed storage cluster storage location. The data writing module is used to write the configuration data on the server side into the distributed storage cluster through a pre-encapsulated data processing interface of the distributed storage cluster; wherein, the data processing interface is obtained by pre-encapsulating the preset interfaces in the distributed storage cluster into multiple types, and the data processing interface includes a data writing interface and a data modification interface. The data reading module is used to respond to the server-side request to read configuration data from the corresponding backend storage and synchronize the configuration data to the backend storage through the data processing interface. The monitoring node module is used to monitor the status of the master node of the distributed storage cluster; wherein, the distributed storage cluster includes a cluster alarm interface; The data backup module is used to select a target master node from the slave nodes and back up the configuration data to the target master node if the master node is detected to be in an abnormal state. The exception notification module is used to send exception notification information to the server using the cluster alarm interface.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the instructions to implement the data read / write method as described in any one of claims 1 to 6.

9. A readable storage medium, characterized in that, A computer program is stored on the readable storage medium, and when the computer program is executed by a processor, it implements the data read / write method as described in any one of claims 1 to 6.

Images