Data storage methods and apparatuses, device, medium and storage cluster

Abstract

The embodiments of the present application relate to the technical field of storage, and disclose data storage methods and apparatuses, a device, a medium and a storage cluster, which can satisfy data storage requirements of nodes in storage clusters. A method comprises: accessing an internal volume of a storage cluster, and writing into the internal volume target data of a node to be stored, wherein the internal volume is created by means of the following step: on the basis of a plurality of configured storage class memory disks of the storage cluster, creating the internal volume, the internal volume being used for storing respective target data of nodes in the storage cluster.

Description

Data storage method, device and equipment, medium and storage cluster

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to a Chinese patent application filed with the Patent Office of China on September 12, 2023, with application number 202311167630.3 and application name “Data Storage Methods, Apparatus and Equipment, Medium and Storage Cluster,” all contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of storage technology, and in particular to a data storage method, apparatus and device, medium and storage cluster. Background Art

[0004] Nodes in a storage cluster need to allocate a large amount of memory to store their own performance monitoring data, metadata, and other large-volume data. However, due to limited memory resources, it is usually impossible to allocate enough memory to nodes to meet their data storage needs.

[0005] Summary of the Invention

[0006] In a first aspect, an embodiment of the present application provides a data storage method, applied to any node in a storage cluster, the method comprising:

[0007] Access the internal volume of the storage cluster and write its own target data to the internal volume for storage;

[0008] The internal volume is created by following the steps below:

[0009] Based on the multiple storage-class memory disks configured in the storage cluster, internal volumes are created. The internal volumes are used to store the target data of each node in the storage cluster.

[0010] In a second aspect, an embodiment of the present application further provides a data storage method, which is applied to an in-memory database configured in a storage cluster, and the method includes:

[0011] In response to receiving a write instruction from a node in the storage cluster, the target data to be written corresponding to the write instruction is directly written into the database table corresponding to the node in its own persistent storage area for storage;

[0012] The persistent storage area of ​​the in-memory database includes an internal volume of the storage cluster created based on multiple storage-level memory disks configured in the storage cluster.

[0013] In a third aspect, an embodiment of the present application further provides a storage cluster, the storage cluster including multiple nodes, wherein:

[0014] A target node among the multiple nodes is used to create an internal volume of the storage cluster based on multiple storage-class memory disks configured in the storage cluster;

[0015] Each of the multiple nodes is used to access the internal volume of the storage cluster and write its own target data into the internal volume for storage.

[0016] In a fourth aspect, an embodiment of the present application further provides a data storage device, which is applied to any node in a storage cluster, and includes:

[0017] A first writing module is configured to access an internal volume of the storage cluster and write its own target data into the internal volume for storage; and

[0018] The internal volume creation module is used to create internal volumes based on multiple storage-level memory disks configured in the storage cluster. The internal volumes are used to store the target data of each node in the storage cluster.

[0019] In a fifth aspect, an embodiment of the present application further provides a data storage device, which is applied to an in-memory database configured in a storage cluster, and the device includes:

[0020] The first response module is configured to respond to receiving a write instruction from a node in the storage cluster.

[0021] The target data to be written corresponding to the write instruction is directly written into the database table corresponding to the node in its own persistent storage area for storage;

[0022] The persistent storage area of ​​the in-memory database includes an internal volume of the storage cluster created based on multiple storage-level memory disks configured in the storage cluster.

[0023] In a sixth aspect, an embodiment of the present application further provides an electronic device comprising a memory, a processor, and computer-readable instructions stored on the memory, wherein the processor executes the computer-readable instructions to implement the data storage method of the first aspect or the second aspect.

[0024] In the seventh aspect, the embodiments of the present application further provide one or more non-volatile computer-readable storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the data storage method of the first aspect or the second aspect is implemented.

[0025] In an eighth aspect, an embodiment of the present application further provides a computer-readable instruction product, comprising computer-readable instructions, which, when executed by a processor, implement the data storage method of the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0027] FIG1 is a flowchart of an implementation of a data storage method provided by one or more embodiments of the present application;

[0028] FIG2 is a schematic diagram of a node accessing an internal volume provided by one or more embodiments of the present application;

[0029] FIG3 is a schematic diagram of another node accessing an internal volume provided by one or more embodiments of the present application;

[0030] FIG4 is a schematic diagram of an in-memory database before and after improvement according to one or more embodiments of the present application;

[0031] FIG5 is a schematic diagram of another node accessing an internal volume provided by one or more embodiments of the present application;

[0032] FIG6 is a flowchart of another data storage method according to one or more embodiments of the present application;

[0033] FIG7 is a schematic structural diagram of a data storage device provided by one or more embodiments of the present application;

[0034] FIG8 is a schematic structural diagram of another data storage device provided by one or more embodiments of the present application;

[0035] FIG9 is a schematic diagram of an electronic device provided by one or more embodiments of the present application;

[0036] FIG10 is a schematic diagram of a computer-readable storage medium provided in one or more embodiments of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0038] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not limited to the listed steps or elements and may include steps or elements that are not listed.

[0039] First, in order to facilitate understanding of the data storage solution of the embodiment of the present application, the following content is introduced.

[0040] 1. Storage Class Memory (SCM) media (i.e., SCM disk)

[0041] SCM media, also known as persistent memory (PM) media or non-volatile memory (NVM), is a very popular new media form in the industry, with the characteristics of persistence and fast byte-level access.

[0042] 2. Dynamic Random Access Memory (DRAM)

[0043] DRAM, the most common system memory, can only retain data for a short time. To retain data, DRAM uses capacitors for storage, so it must be refreshed at regular intervals. If it is not refreshed, the stored information will be lost.

[0044] 3. In-memory database

[0045] An in-memory database is a database that operates directly on data in memory. It takes advantage of the fact that memory reads and writes faster than disks, and that memory is accessed randomly while disks are accessed sequentially, to store data in memory. It mimics the table and index structures in memory and optimizes them for memory characteristics. Compared to traditional databases accessed from disk, in-memory databases are faster.

[0046] 4. Remote Dictionary Server (Redis)

[0047] Redis is an open-source, network-friendly, in-memory, and persistent log-based key-value database written in standard C (ANSI C). It also provides APIs in multiple languages. Like the distributed cache system (memcached), Redis caches data in memory to ensure efficiency. The difference is that Redis periodically writes updated data to disk or writes modifications to appended log files, and implements master-slave synchronization based on this. Redis offers two types of persistence:

[0048] 1) Remote Dictionary Service Database (Redis Database, RDB) persistence: RDB is stored in a specified time interval.

[0049] Write a snapshot of the dataset in memory to disk. During recovery, the snapshot file is read directly into memory. A dataset snapshot can be understood as taking a snapshot of the dataset at a specific point in time and saving it. This allows Redis to back up the current system data at specified intervals or when executing specific commands, writing it to disk in binary format.

[0050] 2) Append-only-file (AOF) persistence: AOF records all executed commands (except read operations) and re-executes them during recovery. If the data is large, it takes a long time to write. AOF defaults to unlimited file appends, and the file size will continue to expand.

[0051] In related technologies, nodes in a storage cluster need to allocate a large amount of memory to store their own performance monitoring data, or metadata (used to record the node's own task execution status) and other large-scale data. However, due to limited memory resources, it is usually impossible to allocate enough memory to the nodes to meet the node's data storage needs.

[0052] Taking performance monitoring data as an example, it provides insights into various performance indicators of current storage devices, helping users understand the operating status of storage devices and enabling storage vendor support personnel to quickly locate issues. Typically, storage devices utilize dual or multiple controllers, and performance monitoring involves the storage and summary display of performance data from each storage device node. Users can expect performance monitoring to last for years, requiring a large number of monitored objects and metrics.

[0053] For example, common monitoring targets include storage arrays, controllers, back-end SAS (Statistics Analysis System) ports, back-end Ethernet ports, front-end FC ports, front-end Ethernet ports, storage pools, hard drives, host groups, Logical Unit Number (LUN) groups, LUNs, and file systems. Taking LUNs as an example, most vendors support thousands of LUNs, with some even supporting up to 65,535. Monitoring target metrics, still using LUNs as an example, typically include Input / Output Operations Per Second (IOPS), Read IOPS, Write IOPS, Average Input / Output (I / O) Response Time, Average Read I / O Response Time, Average Write I / O Response Time, Block Bandwidth, Read Bandwidth, Write Bandwidth, Queue Length, Average Read I / O Size, Average Write I / O Size, Read Cache Hit Rate, Write Cache Hit Rate, Average IO Size, Percentage of Read IOs in Total IOs, and Percentage of Write IOs in Total IOs.

[0054] Due to the long performance monitoring time required by users (possibly up to several years), the large number of monitored objects (some objects may have tens of thousands, such as the number of volumes) and the excessive number of monitoring indicators (for example, there are nearly 20 indicators to monitor for each volume), each node needs to allocate a large amount of memory to store the performance monitoring data generated by the performance monitoring module in the node. However, memory resources are limited, and in most cases it is impossible to allocate enough memory to the performance monitoring module to meet the storage requirements of the performance monitoring data, which in turn leads to a reduction in the number of monitored objects and indicators of the monitoring module, which cannot meet user needs.

[0055] In response to the problems existing in the above-mentioned related technologies, this application proposes a data storage solution that uses lower-cost SCM disks to create internal volumes instead of memory to store large amounts of target data of each node. This can meet the data storage needs of the nodes and achieve persistent storage of target data, thereby reducing the loss of target data.

[0056] The following, in conjunction with the accompanying drawings, describes in detail a data storage method, apparatus and device, medium, and storage cluster provided by the embodiments of the present application through some embodiments and their application scenarios.

[0057] In the first aspect, referring to FIG1 , which is a flowchart of an implementation of a data storage method provided in an embodiment of the present application, the data storage method is applied to any node in a storage cluster, and the method may include the following steps:

[0058] Step S101: Access the internal volume of the storage cluster and write the target data to the internal volume for storage.

[0059] The internal volume is created through the following steps:

[0060] Based on multiple storage-class memory disks configured in the storage cluster, an internal volume is created, and the internal volume is used to store the target data of each node in the storage cluster.

[0061] In specific implementation, when any node in the storage cluster generates target data, it accesses the internal volume of the storage cluster and writes the target data generated by the node to the internal volume. The internal volume (i.e., internal LUN) performs persistent storage of the target data. Compared with the method of allocating memory to achieve storage, the space belonging to the internal volume comes from the SCM disk, which makes the unit cost of the internal volume much lower than the memory. Therefore, when the cost is limited, a larger internal volume can also be set, such as setting an internal volume of several hundred gigabytes (GB) to meet the storage needs of each node in the storage cluster for target data. The internal volume created based on the SCM disk can store the target data for a long time, and there will be no loss of target data due to the node not refreshing the related storage unit in time (such as the node being offline and not refreshing in time). It can also avoid the need to further write the memory data to the hard disk (such as the system disk) to achieve persistent storage, thereby improving storage efficiency. The SCM media access delay is generally less than 1 microsecond (μs), which is 2-3 orders of magnitude faster than the currently commonly used flash memory (NAND Flash), and there is no NAND when reading and writing. The constraints of Flash sequential write and erase-before-write make the operation process simpler. At the same time, SCM media far exceeds NAND Flash in terms of lifespan and data retention. Therefore, memory volumes created based on SCM disks can achieve faster access speeds, more stable data retention, and longer service life than hard drives or memory.

[0062] Users can configure multiple SCM disks for the storage cluster to create internal volumes based on the current storage cluster's storage needs. They can also pre-configure a storage pool containing a sufficient number of SCM disks for the storage cluster and select all or some of the SCM disks in the storage pool to create internal volumes to meet the different data storage needs of the storage cluster. After configuring the SCM disks for the storage cluster, you can set up a script to have the target nodes in the storage cluster (such as the master node or other designated nodes) run the script after installation to automatically create the storage cluster's internal volumes based on the SCM disks. After the internal volumes are created, each node in the storage cluster can write its own target data to the internal volumes for persistent storage.

[0063] It can be seen from the above technical solution that, considering that the unit cost of storage-level memory disks is much lower than that of memory, multiple storage-level memory disks can be configured for the storage cluster, and internal volumes can be created for the storage cluster based on these multiple storage-level memory disks to replace memory to store large amounts of target data for each node. This can meet the data storage needs of the nodes and achieve persistent storage of target data, thereby reducing the loss of target data.

[0064] Optionally, in one embodiment, the method further comprises:

[0065] Access the internal volume of the storage cluster and read the target data of one or more nodes in the storage cluster from the internal volume.

[0066] In a specific implementation, a node can access the internal volume to read its own target data or the target data of other nodes, thereby eliminating the communication bandwidth occupied by the transmission of target data (such as metadata, performance monitoring data, etc.) between nodes. For example, the nodes of a traditional storage cluster need to communicate with each other to synchronize metadata to ensure that information related to the task execution status of the node is not lost after the node goes offline. However, the embodiment of the present application creates an internal volume to store the metadata of each node, so that after the node goes offline and recovers, it can directly obtain its own metadata from the internal volume, thereby eliminating the communication bandwidth occupied by the transmission of metadata between nodes.

[0067] As a possible implementation, the target data includes performance monitoring data, and the node is a master node in the storage cluster or a new master node after a master-slave node switch; accessing an internal volume of the storage cluster and reading target data of one or more nodes in the storage cluster from the internal volume includes:

[0068] In response to receiving the target query request, reading the performance monitoring data of each node in the storage cluster from the internal volume;

[0069] The method further includes:

[0070] Based on the performance monitoring data of each node in the storage cluster, the query results for the target query request are determined and returned.

[0071] It is understandable that, considering that traditional storage clusters usually set up a master node (i.e., master controller) and one or more slave nodes (i.e., slave controllers) from each node, each node will allocate a piece of memory to store its own performance monitoring data, and the slave node will regularly send its own performance monitoring data to the master node. The master node summarizes and calculates the performance monitoring data of each node to obtain the performance monitoring data of the entire storage cluster, and writes the summarized and calculated performance monitoring data to the local system disk for persistent storage. When an external target query request is received, the summarized and calculated performance monitoring data is returned to the outside as a query result. Each node has its own memory and system disk (for storing system installation files).

[0072] However, since the slave nodes need to send their own performance monitoring data to the master node regularly (shortly, mostly 5 seconds), a huge amount of performance monitoring data needs to be frequently sent between nodes. The communication bandwidth between nodes is frequently occupied by performance monitoring data, which affects the transmission of redundant mirror data such as cache between nodes, and ultimately affects the performance of the storage cluster.

[0073] As shown in Figure 2, in order to solve the problem of performance monitoring data occupying the communication bandwidth between nodes, an embodiment of the present application adopts an internal volume created based on the SCM disk to persistently store the performance monitoring data of the master node and the slave node in the storage cluster. After receiving a target query request sent from the outside (such as a user), the graphical user interface (GUI) process in the master node will directly read the performance monitoring data of the node related to the target query request from the internal volume, summarize and calculate the read data to obtain the query result, and display the query result to the outside. This can avoid the huge bandwidth consumption caused by the slave node synchronizing the performance monitoring data with the master node, thereby ensuring the performance of the storage cluster.

[0074] Considering that the storage cluster may also send the controller in the storage cluster to switch between master and slave nodes due to software or hardware reasons, that is, a slave node in the storage cluster becomes the new master node. Since the traditional storage cluster synchronizes performance monitoring data from each slave node to the master node, after the master-slave node switch occurs in the storage cluster, the new master node only has its own performance monitoring data, that is, the new master node needs to re-collect the performance monitoring data of each slave node for summary calculation, and then store the summarized performance monitoring data to the local system disk. This will cause performance monitoring data anomalies for a period of time, and the new master node cannot normally process the target query request during the data collection period.

[0075] In order to solve the problem of needing to resynchronize performance monitoring data after the master-slave control node switches, an embodiment of the present application adopts an internal volume created based on the SCM disk to persistently store the performance monitoring data of the master control node and the slave control node in the storage cluster. After receiving the target query request after the master-slave control node switches, the new master control node can directly access the internal volume to obtain the performance monitoring data of each slave control node in the storage cluster, so as to normally perform operations such as summary calculation of performance monitoring data and return of query results. This makes it unnecessary for the storage cluster to resynchronize the performance monitoring data after the master-slave control node switches, and can avoid performance monitoring data anomalies and the inability to process target query requests normally due to the master-slave control node switching, thereby improving the performance monitoring performance of the storage cluster.

[0076] Optionally, in one embodiment, in the above step S101, accessing the internal volume of the storage cluster and writing the own target data into the internal volume for storage includes:

[0077] Through the interface provided by the database software installed on the node, access the internal volume of the storage cluster and write your own target data into the database table corresponding to the node for storage.

[0078] The internal volume is used to store the database tables corresponding to the respective nodes in the storage cluster, and a single database table is used to store the target data of a single node.

[0079] In the specific implementation, considering that it is too cumbersome to organize and manage the target data by the nodes, the embodiment of the present application introduces database technology to organize and manage the target data of each node. As shown in Figure 3, each node directly writes data to the data table corresponding to each node on the internal volume through the interface provided by the database software. The database (i.e., database software) writes the target data of each node to the internal volume for storage in the form of the database table corresponding to each node (which is an object used to store data in the database and is a collection of structured data), thereby realizing the organization and management of the target data of each node.

[0080] As a possible implementation, the method further includes:

[0081] During the node system installation phase, install the database software;

[0082] The database software is used to run an in-memory database, which performs data operations and stores target data in an internal volume.

[0083] It is understandable that, considering that the performance monitoring data of the node needs to be updated and recorded in real time, directly using a traditional database will result in long waiting times for writing and reading, which will have a certain impact on the performance of the storage cluster and will lead to a poor user experience (such as data not being updated on the GUI for a long time). Therefore, the embodiment of the present application selects an in-memory database (such as eXtremeDB, SolidDB, Oracle TimesTen, Redis and other in-memory databases) to organize and manage performance monitoring data, and improves the traditional in-memory database by using the internal volume as the database's data persistent storage area, so that the database can operate and persistently store data in the SCM disk.

[0084] In specific implementation, the database software installation package corresponding to the in-memory database can be integrated into the system installation file. Each node in the storage cluster installs the in-memory database during the system installation phase, and creates an internal volume based on the SCM disk according to the configured script. The created internal volume is configured as the persistent storage area of ​​the in-memory database. Each node can then directly implement data operations such as adding, deleting, modifying, and querying the target data in the internal volume through the interface provided by the in-memory database installed by itself, as well as persistent storage of the target data.

[0085] As shown in Figure 4, the embodiment of the present application breaks the pattern in which data in traditional memory databases such as Redis can only be read and written on volatile storage (such as DRAM). The database data can be read and written directly on a high-performance SCM disk (equivalent to using an SCM disk instead of DRAM), and the SCM disk can be used to directly persist the data in the database. There is no need to persist the data in the database to the hard disk through RDB or AOF operations, which can improve the storage and access efficiency of the target data.

[0086] Optionally, in one embodiment, the creation of an internal volume based on multiple storage-class memory disks configured in the storage cluster includes:

[0087] Build a redundant array of disks based on multiple storage-class memory disks configured in the storage cluster;

[0088] Create internal volumes based on redundant disk arrays.

[0089] In this embodiment, as shown in FIG5 , each SCM disk can be organized into a redundant disk array based on a storage solution such as RAID 5, and then an internal volume can be created based on the redundant disk array. This can avoid loss of stored target data due to damage to the SCM disk, thereby improving the storage security of the target data.

[0090] As a possible implementation, the above-mentioned creation of an internal volume based on a redundant array of disks includes:

[0091] Create a single internal volume based on redundant array of disks;

[0092] Divide the internal volume into storage partitions corresponding to each node in the storage cluster. One storage partition is used to store the target data of one node.

[0093] The method further includes:

[0094] Access the storage partitions corresponding to other nodes in the storage cluster in read-only mode.

[0095] In specific implementation, a larger internal volume can be created based on the disk array or disk redundant array formed by the SCM disk to store the target data of all nodes in the storage cluster. The internal volume supports parallel access from multiple nodes. A node can read and write its own target data from its own corresponding storage partition, and read the target data of other nodes from the storage partitions corresponding to other nodes in read-only mode, thereby avoiding the situation where the data in the storage partition corresponding to the node itself is modified by other nodes, thereby further improving the storage security of the target data.

[0096] As another possible implementation, the above-mentioned creation of an internal volume based on a redundant array of disks includes:

[0097] Based on the redundant disk array, create internal volumes corresponding to each node in the storage cluster. One internal volume is used to store the target data of one node.

[0098] The method further includes:

[0099] Access the internal volumes corresponding to other nodes in the storage cluster in read-only mode.

[0100] In specific implementation, based on the disk array or disk redundant array formed by the SCM disk, an internal volume can be created for each node to store the target data of each node respectively, that is, several internal volumes are created for several nodes, and the nodes access their own internal volumes to read and write their own target data, thereby improving the parallel performance of multiple nodes accessing the internal volumes. Any node can also use read-only access to the internal volumes corresponding to other nodes to read the target data of other nodes, which can avoid bandwidth consumption caused by transmitting target data between nodes, and can also avoid the situation where the data in the internal volume corresponding to the node itself is modified by other nodes, thereby further improving the storage security of the target data.

[0101] Based on the above embodiments, the present application improves the in-memory database based on the SCM medium, and uses the improved in-memory database to organize and manage the target data in the storage cluster. Each node writes its own target data into the SCM medium for storage through the interface provided by the in-memory database, which can meet the node's storage requirements for target data, especially the storage requirements for large amounts of performance monitoring data, and can further meet the user's growing requirements for the number of performance monitoring objects, the number of performance monitoring object indicators, the monitoring duration, etc., and because each node can see and access the internal volume formed based on the SCM medium, after each node writes its own target data into the internal volume, any node can access the performance monitoring data stored in the internal volume of each node through the database interface, thereby avoiding frequent transmission of target data between nodes and occupying valuable inter-node bandwidth, thereby improving the performance of the storage cluster.

[0102] In the second aspect, referring to FIG6 , which is a flowchart of another data storage method provided in an embodiment of the present application, the data storage method is applied to an in-memory database configured in a storage cluster. The method may include the following steps:

[0103] Step S201: In response to receiving a write instruction from a node in the storage cluster, the target data to be written corresponding to the write instruction is directly written into the database table corresponding to the node in its own persistent storage area for storage.

[0104] The persistent storage area of ​​the in-memory database includes an internal volume of the storage cluster created based on multiple storage-level memory disks configured in the storage cluster.

[0105] Users can configure multiple SCM disks for the storage cluster to create internal volumes based on the storage needs of the current storage cluster. They can also pre-configure a storage pool containing a sufficient number of SCM disks for the storage cluster and select all or part of the SCM disks in the storage pool to create internal volumes to meet the different data storage needs of the storage cluster. After configuring the SCM disks for the storage cluster, you can set up a script to have the target nodes in the storage cluster (such as the master node or other designated nodes) run the script after installation to automatically create the internal volumes of the storage cluster based on the SCM disks. After the internal volumes are created, each node in the storage cluster can write its own target data to the internal volumes for persistent storage through the interface provided by the in-memory database.

[0106] As a possible implementation, the method further includes:

[0107] In response to receiving a read instruction from a node in the storage cluster, selecting target data to be read corresponding to the read instruction from the in-memory database;

[0108] Returns the target data to be read to the node.

[0109] As a possible implementation, the persistent storage area includes internal volumes corresponding to each node in a storage cluster created based on multiple storage-level memory disks, and one internal volume is used to store a database table corresponding to one node.

[0110] As a possible implementation, the persistent storage area includes a single internal volume created based on multiple storage-level memory disks. The single internal volume includes storage partitions corresponding to each node in the storage cluster. One storage partition is used to store a database table corresponding to one node.

[0111] As a possible implementation, the internal volume is created based on a redundant array of disks constructed by multiple storage-class memory disks.

[0112] It can be seen from the above technical solution that, considering that the unit cost of storage-level memory disks is much lower than that of memory, multiple storage-level memory disks can be configured for the storage cluster, and internal volumes can be created for the storage cluster based on these multiple storage-level memory disks to replace memory to store large amounts of target data for each node. This can meet the data storage needs of the nodes and achieve persistent storage of target data, thereby reducing the loss of target data.

[0113] In a third aspect, an embodiment of the present application provides a storage cluster, the storage cluster including multiple nodes, wherein:

[0114] A target node among the multiple nodes is used to create an internal volume of the storage cluster based on multiple storage-class memory disks configured in the storage cluster;

[0115] Each of the multiple nodes is used to access the internal volume of the storage cluster and write its own target data into the internal volume for storage.

[0116] Optionally, each of the multiple nodes is further configured to access an internal volume of the storage cluster and read target data of itself and / or other nodes from the internal volume, where the target data includes at least one of metadata and performance monitoring data.

[0117] Optionally, the target data includes performance monitoring data, and the multiple nodes include a master control node and at least one slave control node, wherein:

[0118] The master node or the new master node after the master-slave node switches is used to respond to receiving a target query request, read the performance monitoring data of itself and the slave node from the internal volume, and determine and return the query result for the target query request based on the read performance monitoring data.

[0119] Optionally, the target node is further configured to form a redundant disk array based on multiple storage-level memory disks configured in the storage cluster, and to create an internal volume based on the redundant disk array.

[0120] Optionally, the target node is further configured to create internal volumes corresponding to respective nodes in the storage cluster based on a redundant disk array, where one internal volume is used to store target data of one node;

[0121] Each of the multiple nodes is further configured to access the internal volumes corresponding to other nodes in the storage cluster in a read-only manner.

[0122] Optionally, the target node is further configured to create a single internal volume based on the redundant array of disks, and to divide the internal volume into storage partitions corresponding to respective nodes in the storage cluster, wherein one storage partition is used to store target data of one node;

[0123] Each of the multiple nodes is also used to access the storage partitions corresponding to other nodes in the storage cluster in a read-only manner.

[0124] It can be seen from the above technical solution that, considering that the unit cost of storage-level memory disks is much lower than that of memory, multiple storage-level memory disks can be configured for the storage cluster, and internal volumes can be created for the storage cluster based on these multiple storage-level memory disks to replace memory to store large amounts of target data for each node. This can meet the data storage needs of the nodes and achieve persistent storage of target data, thereby reducing the loss of target data.

[0125] In a fourth aspect, an embodiment of the present application provides a data storage device, which is applied to any node in a storage cluster. As shown in FIG7 , the device includes:

[0126] The first writing module is used to access the internal volume of the storage cluster and write its own target data into the internal volume for storage;

[0127] The internal volume creation module is used to create internal volumes based on multiple storage-level memory disks configured in the storage cluster. The internal volumes are used to store the target data of each node in the storage cluster.

[0128] Optionally, the device further comprises:

[0129] The first reading module is configured to access an internal volume of the storage cluster and read target data of one or more nodes in the storage cluster from the internal volume.

[0130] Optionally, the target data includes performance monitoring data, and the node is a master node in the storage cluster or a new master node after a master-slave node switch;

[0131] The first reading module includes:

[0132] A first reading submodule is configured to read the performance monitoring data of each node in the storage cluster from the internal volume in response to receiving a target query request;

[0133] The device also includes:

[0134] The result return module is used to determine and return the query result for the target query request based on the performance monitoring data of each node in the storage cluster.

[0135] Optionally, the first writing module includes:

[0136] The first writing submodule is used to access the internal volume of the storage cluster through the interface provided by the database software installed on the node, and write its own target data into the database table corresponding to the node for storage;

[0137] The internal volume is used to store the database tables corresponding to each node in the storage cluster, and a single database table is used to store the target data of a single node.

[0138] Optionally, the device further comprises:

[0139] The software installation module is used to install the database software during the system installation phase of the node;

[0140] The database software is used to run an in-memory database, and the in-memory database is used to perform data operations and store target data in an internal volume.

[0141] Optionally, the internal volume creation module includes:

[0142] A first creation submodule is used to build a redundant array of disks based on multiple storage-class memory disks configured in the storage cluster;

[0143] The second creation submodule is used to create an internal volume based on the redundant disk array.

[0144] Optionally, the second creation submodule includes:

[0145] The third creation submodule is used to create internal volumes corresponding to each node in the storage cluster based on the redundant disk array. One internal volume is used to store target data of one node.

[0146] The device also includes:

[0147] The first access module is used to access the internal volumes corresponding to other nodes in the storage cluster in a read-only manner.

[0148] Optionally, the second creation submodule includes:

[0149] A fourth creation submodule is used to create a single internal volume based on the redundant array of disks;

[0150] The region partitioning module is used to divide the internal volume into storage partitions corresponding to each node in the storage cluster. One storage partition is used to store the target data of one node.

[0151] The device also includes:

[0152] The second access module is used to access the storage partitions corresponding to other nodes in the storage cluster in a read-only manner.

[0153] It can be seen from the above technical solution that, considering that the unit cost of storage-level memory disks is much lower than that of memory, multiple storage-level memory disks can be configured for the storage cluster, and internal volumes can be created for the storage cluster based on these multiple storage-level memory disks to replace memory to store large amounts of target data for each node. This can meet the data storage needs of the nodes and achieve persistent storage of target data, thereby reducing the loss of target data.

[0154] In a fifth aspect, an embodiment of the present application provides another data storage device, which is applied to an in-memory database configured in a storage cluster. As shown in FIG8 , the device includes:

[0155] A first response module is configured to, in response to receiving a write instruction from a node in the storage cluster, directly write the target data to be written corresponding to the write instruction into the database table corresponding to the node in its own persistent storage area for storage;

[0156] The persistent storage area of ​​the in-memory database includes an internal volume of the storage cluster created based on multiple storage-level memory disks configured in the storage cluster.

[0157] Optionally, the device further comprises:

[0158] A second response module is configured to, in response to receiving a read instruction from a node in the storage cluster, select target data to be read corresponding to the read instruction from the memory-based database;

[0159] The first returning module is used to return the target data to be read to the node.

[0160] Optionally, the persistent storage area includes internal volumes corresponding to respective nodes in a storage cluster created based on multiple storage-level memory disks, and one internal volume is used to store a database table corresponding to one node.

[0161] Optionally, the persistent storage area includes a single internal volume created based on multiple storage-level memory disks. The single internal volume includes storage partitions corresponding to each node in the storage cluster. One storage partition is used to store a database table corresponding to one node.

[0162] Optionally, the internal volume is created based on a redundant array of disks constructed from multiple storage-class memory disks.

[0163] It can be seen from the above technical solution that, considering that the unit cost of storage-level memory disks is much lower than that of memory, multiple storage-level memory disks can be configured for the storage cluster, and internal volumes can be created for the storage cluster based on these multiple storage-level memory disks to replace memory to store large amounts of target data for each node. This can meet the data storage needs of the nodes and achieve persistent storage of target data, thereby reducing the loss of target data.

[0164] It should be noted that the device embodiment is similar to the method embodiment, so the description is relatively simple, and the relevant parts can be referred to the method embodiment.

[0165] The present application also provides an electronic device, as shown in FIG9 , which is a schematic diagram of the electronic device provided in the present application. As shown in FIG9 , the electronic device 100 includes a memory 110 and a processor 120 . The memory 110 and the processor 120 are connected via a bus. The memory 110 stores computer-readable instructions that can be executed on the processor 120 to implement the steps of the data storage method disclosed in the present application.

[0166] The present application also provides one or more non-volatile computer-readable storage media storing computer-readable instructions. See Figure 10 for a computer-readable storage medium, which is a schematic diagram of the computer-readable storage medium provided in the present application. As shown in Figure 10, computer-readable storage medium 200 stores computer-readable instructions 210, which, when executed by a processor, implement the steps of the data storage method disclosed in the present application.

[0167] An embodiment of the present application also provides a computer-readable instruction product, including computer-readable instructions, which, when executed by a processor, implement the steps in the data storage method disclosed in the embodiment of the present application.

[0168] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0169] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer-readable instruction products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application can take the form of a computer-readable instruction product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, systems, devices, storage media, and program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer-readable instructions. These computer-readable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0171] These computer-readable instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0172] These computer-readable instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0173] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0174] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0175] The above is a detailed introduction to a data storage method, device and equipment, medium and storage cluster provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core ideas. At the same time, for general technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data storage method, characterized in that: Applied to any node in a storage cluster, the method includes: Access the internal volume of the storage cluster and write its own target data into the internal volume for storage; The internal volume is created by the following steps: The internal volume is created based on a plurality of storage-level memory disks configured in the storage cluster, and the internal volume is used to store target data of each node in the storage cluster.

2. The method according to claim 1, characterized in that The method further comprises: An internal volume of the storage cluster is accessed, and target data of one or more nodes in the storage cluster is read from the internal volume.

3. The method according to claim 2, characterized in that The target data includes performance monitoring data, and the node is a master control node in the storage cluster or a new master control node after a master-slave control node is switched; The accessing the internal volume of the storage cluster and reading target data of one or more nodes in the storage cluster from the internal volume includes: In response to receiving the target query request, reading the performance monitoring data of each node in the storage cluster from the internal volume; The method further comprises: According to the performance monitoring data of each node in the storage cluster, a query result for the target query request is determined and returned.

4. The method according to claim 1, characterized in that The accessing the internal volume of the storage cluster and writing the own target data into the internal volume for storage includes: Access the internal volume of the storage cluster through the interface provided by the database software installed on the node, and write its own target data into the database table corresponding to the node for storage; The internal volume is used to store database tables corresponding to the respective nodes in the storage cluster, and a single database table is used to store target data of a single node.

5. The method according to claim 4, characterized in that The method further comprises: During the system installation phase of the node, installing the database software; The database software is used to run an in-memory database, and the in-memory database is used to perform data operations and storage on the target data in the internal volume.

6. The method according to any one of claims 1 to 5, characterized in that: The step of creating the internal volume based on the multiple storage-level memory disks configured in the storage cluster includes: Building a redundant array of disks based on the multiple storage-class memory disks configured in the storage cluster; and The internal volume is created based on the redundant array of disks.

7. The method according to claim 6, characterized in that The step of creating the internal volume based on the redundant array of disks includes: Based on the redundant array of disks, creating internal volumes corresponding to respective nodes in the storage cluster, one internal volume being used to store target data of one of the nodes; The method further comprises: Access the internal volumes corresponding to other nodes in the storage cluster in a read-only manner.

8. The method according to claim 6, characterized in that The step of creating the internal volume based on the redundant array of disks includes: Creating a single internal volume based on the redundant array of disks; and Dividing storage partitions corresponding to respective nodes in the storage cluster from the internal volume, one storage partition is used to store target data of one node; The method further comprises: Access the storage partitions corresponding to other nodes in the storage cluster in a read-only manner.

9. A data storage method, characterized in that: Applied to an in-memory database configured by a storage cluster, the method comprises: In response to receiving a write instruction from a node in the storage cluster, directly writing the target data to be written corresponding to the write instruction into the database table corresponding to the node in its own persistent storage area for storage; The persistent storage area of ​​the memory-type database includes an internal volume of the storage cluster created based on a plurality of storage-level memory disks configured in the storage cluster.

10. The method according to claim 9, characterized in that The method further comprises: In response to receiving a read instruction from a node in the storage cluster, selecting target data to be read corresponding to the read instruction from the in-memory database; and The target data to be read is returned to the node.

11. The method according to claim 9, characterized in that The persistent storage area includes internal volumes corresponding to each node in the storage cluster created based on the multiple storage-level memory disks, and one of the internal volumes is used to store a database table corresponding to one of the nodes.

12. The method according to claim 9, characterized in that The persistent storage area includes a single internal volume created based on the multiple storage-level memory disks, and the single internal volume includes storage partitions corresponding to each node in the storage cluster, and one storage partition is used to store a database table corresponding to one node.

13. The method according to any one of claims 9 to 12, characterized in that: The internal volume is created based on a redundant array of disks constructed by the multiple storage-level memory disks.

14. A storage cluster, characterized in that: The storage cluster includes multiple nodes, wherein: A target node among the multiple nodes is used to create an internal volume of the storage cluster based on multiple storage-level memory disks configured in the storage cluster; Each node among the multiple nodes is used to access the internal volume of the storage cluster and write its own target data into the internal volume for storage.

15. The storage cluster according to claim 14, characterized in that: Each of the multiple nodes is further configured to access an internal volume of the storage cluster and read target data of itself and / or other nodes from the internal volume, wherein the target data includes at least one of metadata and performance monitoring data.

16. The storage cluster according to claim 15, characterized in that: The target data includes performance monitoring data, and the multiple nodes include a master control node and at least one slave control node, wherein: The master control node or the new master control node after the master-slave control node switches is used to respond to receiving a target query request, read the performance monitoring data of itself and the slave control node from the internal volume, and determine and return the query result for the target query request based on the read performance monitoring data.

17. The storage cluster according to any one of claims 14 to 16, characterized in that: The target node is further used to form a disk redundant array based on multiple storage-level memory disks configured in the storage cluster, and to create the internal volume based on the disk redundant array.

18. The storage cluster according to claim 17, characterized in that: The target node is further used to create internal volumes corresponding to each node in the storage cluster based on the redundant array of disks, and one internal volume is used to store target data of one node; Each of the multiple nodes is further configured to access the internal volumes corresponding to other nodes in the storage cluster in a read-only manner.

19. The storage cluster according to claim 17, characterized in that: The target node is further used to create a single internal volume based on the redundant array of disks, and to divide the internal volume into storage partitions corresponding to each node in the storage cluster, wherein one storage partition is used to store target data of one node; Each of the multiple nodes is further configured to access storage partitions corresponding to other nodes in the storage cluster in a read-only manner.

20. A data storage device, characterized in that: Applied to any node in a storage cluster, the device comprises: A first writing module is used to access the internal volume of the storage cluster and write its own target data into the internal volume for storage; and The internal volume creation module is used to create the internal volume based on the multiple storage-level memory disks configured in the storage cluster, and the internal volume is used to store the target data of each node in the storage cluster.

21. A data storage device, characterized in that: Applicable to a memory database configured by a storage cluster, the device comprises: A first response module is used for, in response to receiving a write instruction from a node in the storage cluster, directly writing the target data to be written corresponding to the write instruction into a database table corresponding to the node in its own persistent storage area for storage; The persistent storage area of ​​the memory-type database includes an internal volume of the storage cluster created based on a plurality of storage-level memory disks configured in the storage cluster.

22. An electronic device comprising a memory, a processor, and computer-readable instructions stored in the memory, characterized in that: The processor executes the computer-readable instructions to implement the data storage method according to any one of claims 1 to 8 or to implement the data storage method according to any one of claims 9 to 13.

23. One or more non-volatile computer-readable storage media storing computer-readable instructions, wherein when the computer-readable instructions are executed by one or more processors, the one or more processors execute the data storage method as described in any one of claims 1 to 8 or implement the data storage method as described in any one of claims 9 to 13.

Images