54 results about "Clustered file system" patented technology

Providing issue resolution in a cluster system by monitoring system operation to detect occurrence of a system error, and automatically generating, upon detection of the error condition, a core file for a user node. The core file captures a current memory state of a respective node, where the current memory state comprises system statistics, system information, and logs. An intelligent core debugger extracts information from a core file to generate a core file report that is sent to a vendor for a quick determination of whether sufficient information is in the report to allow the vendor to recommend a fix, or whether further information from is required, including the core file itself, if necessary. This prevents the need to send an entire core file to a vendor in every instance of a system fault.

A multi-node, multi-container cluster system that generates, aggregates, and manages log files from services and components includes a serviceability suite of functions for the system as it performs its tasks of supporting applications for a user. A logging function records events and states of nodes as the applications are run, an audit log processing function processes and compiles the logs for audit purposes, and the relevant log data for is provided in the form of uniform core files. Failure processing functions and debugging tools are provided to help analyze, debug, and correct issues that may be encountered. A support bundle processing function is provided to generate and process the log, debugging, failure, and other information for transfer to vendors or other third parties tasked with supporting the cluster network.

The embodiment of the invention discloses an interaction method and device for an OCFS2 cluster service and a block storage service, and the method comprises the steps: receiving a volume operation request through the block storage service, carrying out the analysis of the volume operation request, and determining a request type and a resource identifier; initiating a corresponding lock request to a distributed lock manager of any OCFS2 cluster service through the block storage service according to the request type; performing lock conflict detection on the lock request through the distributed lock manager, and granting a corresponding lock permission to the target resource when a lock conflict detection result is no conflict; executing a corresponding target operation on the target resource through the block storage service according to the lock permission and the resource identifier; according to the scheme, the problem that the interaction reliability of the cloud operating system is low is solved, the OCFS2 cluster service of any node can execute the target operation on the target resource in the OCFS2 cluster file system based on the volume operation request, normal interaction of other nodes is not affected by a single node fault, and the interaction reliability in the cloud operating system is remarkably improved.

A telemetry processing system in a cluster network generates telemetry data from a plurality of telemetry producers and formats it into a structured format for storage. Producers define new metrics for use in a running network. Role-based access control (RBAC) levels for metrics are mapped to users based on roles within an organization using the produced telemetry data. A final RBAC level of a new metric is established through a recommendation and validation process between the producer and an RBAC recommendation module. Once validated, the final RBAC level and new metric are stored in a telemetry catalog. Telemetry datasets for the new metric are sent for storage and transmitted to appropriate users based on RBAC rules including the final RBAC level.

A telemetry processing system in a cluster network receives telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. A telemetry producer can identify consumers to receive a new metric dataset as part of the telemetry data. This data may include sensitive data subject to compliance regulations with respect to data security and privacy. A telemetry collector validates schema of the new metric dataset and a list of consumers through an acceptance or rejection decision. If accepted, the listed consumers can receive the new metric dataset, otherwise they are blocked. This prevents the exposure of sensitive data to unvetted consumers.

A computer-implemented method, according to one approach, includes: receiving a request to create a new node in a clustered filesystem. In response to receiving the request, a copy of details is retrieved, the copy of details being associated with an established singleton cluster preconfigured to access the clustered filesystem. The retrieved copy of details are further applied to a new node in the clustered filesystem. Moreover, data structures of the new node are modified according to the received request. Other systems, methods, and computer program products are described in additional approaches herein.

The invention provides a cluster file system of a database. Wherein the cluster file system of the database comprises an application layer, a proxy distribution layer, a hash ring layer, a file system layer and a shared storage layer, operation on object data can be distributed through the proxy distribution layer, the same object data is only processed at a certain physical node, lock resources are prevented from being scrambled, and the operation efficiency is improved. Meanwhile, each physical node caches the corresponding partition resource group lock into the memory of each physical node, and lock switching is not carried out when needed, so that the time delay of lock switching is reduced; therefore, each physical node caches the metadata of the object data of the corresponding partition in the memory of each physical node, the metadata is not obtained from a disk when needed, and the time delay of disk IO is reduced.

The present application relates to the field of cloud computing, and discloses a cluster file system testing method, device, and medium, including: obtaining a test information file, the test information file including at least target physical machine and virtualization platform information, to provide technical support for subsequent testing work. Deploy a virtualization platform in the target physical machine, deploy a test cluster in the virtualization platform, and test the cluster file system in each test cluster to determine whether the cluster file system meets the first performance requirement. If the cluster file system meets the first performance requirement, test the cluster file system in the virtualization platform to determine whether the cluster file system meets the second performance requirement; if the second performance requirement is met, the system is stable. It can be seen that in this application, a test cluster is constructed by using nested virtual machines in the virtualization platform in the physical machine, thereby comprehensively testing the cluster file system to detect the stability of the system when running on the virtualization platform.

File names and inodes are indexed directly in a filesystem namespace consisting of a layered set of key value store (KVS) database logical tables. The KVS is used directly for organizing file and directory metadata on a cluster file system instead of traditional Btree structures. Embodiments provide file deletion in the file system namespace, and other features include snapshot deletion and cleaning the namespace. A name hash index is used to speed filename lookup in the file system namespace. Two key types of INODE and DIRENT are used to encode namespace metadata using self-identify variable length records in the namespace. Point-in-time snapshot copies are also formalized using KVS database operations as well operations on sets of snapshots for operations including garbage collection and data replication.

A system can, for a cluster that comprises a group of nodes and that hosts a group of virtual machines, create and attach respective new virtual network interface cards to respective virtual machines of the group of virtual machines, wherein the respective new virtual network interface cards are assigned to the new external subnet. The system can create the new external subnet. The system can delete respective current virtual network interface cards from the respective virtual machines, wherein the respective current virtual network interface cards are assigned to the current external subnet. The system can acquire respective dynamic host control protocol leases for the respective virtual machines, wherein the respective dynamic host control protocol leases correspond to the new external subnet. The system can halt a connection from a computer on the new external subnet via the respective new virtual network interface cards.

A key value store is maintained in which the key value store includes pages of a B+ Tree structure holding file metadata of a logical partition of a filesystem. The pages are associated with keys through which the pages are retrieved from the key value store. The keys are named. A clone is created of the logical partition, the logical partition now being a parent and the clone now being a child. The naming of the keys is used to allow the pages stored in the key value store to be read by any of the parent or child. The naming of the keys is used to allow any of the parent or child to make updates to the pages, where updated pages are owned by a respective parent or child that made the updates, and pages not updated remain accessible by any of the parent or child.

A computer-implemented method for provisioning cloud computing clusters includes receiving a request to create a cloud computing cluster, the cloud computing cluster comprising a clustered filesystem and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster and initiating an initialization process for a single processing node and a corresponding attached storage device responsive to receiving the request to create the cloud computing cluster. The method may also include, previous to completion of the initialization process, requesting and receiving an IP address for the single processing node and a device ID for the corresponding attached storage device from one or more cloud infrastructure controllers and configuring the clustered filesystem and a corresponding WAN cache using the received IP address and the received device ID. A system and computer program product corresponding to the above method are also disclosed herein.

Forechannel connections are established from client-side deduplication libraries to a namespace node of a set of namespace nodes formed as a cluster. The namespace nodes are responsible for namespace operations on files managed by a deduplication filesystem. The forechannel connections are responsible for carrying file data between the client-side deduplication libraries and the namespace node. Server-initiated connections are established from the namespace node to the client-side deduplication libraries. The server-initiated connections are responsible for carrying instructions from the namespace node to the client-side deduplication libraries. While a transfer of file data is in progress, the client-side deduplication libraries are instructed over the server-initiated connections to failover the transfer to one or more other namespace nodes. After the failover, the namespace node is upgraded.

A computer-implemented method for provisioning a cloud computing cluster, comprising: receiving a request to create a cloud computing cluster, the cloud computing cluster comprising a clustered file system and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster; and starting an initialization process for the single processing node and the corresponding attached storage device in response to receiving a request for creating the cloud computing cluster. The method may also include requesting and receiving an IP address for the single processing node and a device ID for the corresponding attached storage device from the one or more cloud infrastructure controllers prior to completing the initialization process; and configuring the clustered file system and the corresponding WAN cache using the received IP address and the received device ID. Systems and computer program products corresponding to the above method are also disclosed herein.

A process of recovering from a failure of a solid-state device (SSD) storing a persistent volume (PV) for logging in a cluster network. The PV is recreated on a different SSD that has spare capacity and IOPS resources, and log information is redirected from an application to the relocated PV on the different SSD. Upon replacement of the original failed SSD, the PV can be copied from the different SSD to the new replacement SSD, or the replacement SSD can be left empty, and the relocated PV on the different SSD can continue to be used. This process provides a high degree of redundancy with regard to logging so that a failure of single SSD does not bring down the node or the entire cluster network.

The present application provides a sequential read prefetching method and apparatus based on an Inspur Cluster File System (ICFS) distributed block storage system, a device, and a non-volatile readable storage medium. The method includes: in response to an OSD receiving a read request issued by a client, determining whether the read request is a sequential read request; in response to the read request being the sequential read request, creating a prefetching sliding window according to a data object to be read in the read request; in response to completion of the creation of the prefetching sliding window, calculating, according to the data object to be read in the read request, anew volume object needing to be prefetched; and adding the calculated new volume object needing to be prefetched into a queue of objects to be prefetched of the prefetching sliding window and executing prefetching.

The technology described herein is directed towards deploying a node cluster, e.g., in a cloud environment, based on abstract user storage solution inputs / service level objective input data, including user-specified storage capacity and performance characteristics. An intelligent system converts the input data into cluster topology data (e.g., virtual machines, storage devices / volumes and other information) that can be used for initial cluster deployment or infrastructure updates to existing node clusters in multi-cloud environments. The input data can be directed to an initial deployment of a new node cluster or an update to an existing node cluster. Further, the system can take customer rules as input data, monitor cluster conditions corresponding to those rules, and update the cluster based on a condition in the rules being met, by taking an action associated with the rule's condition to automatically generate and handle a cluster update request.

A telemetry processing system in a cluster network receives telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. One or more consumers subscribe to receive respective data of the telemetry data. A selected transport interface transmits the appropriate telemetry datasets to subscribed consumers. The system automatically updates the subscriptions of the consumers to conform any update to the telemetry data, the telemetry producers, or transport mechanisms.

This invention discloses a method, apparatus, and medium for accessing a clustered file system, applicable to virtualization and hyper-converged infrastructure. Based on block file data I / O operations, the Paxos algorithm is applied to determine the target node for lock request access. The entire process achieves node-to-node access without relying on the network. I / O operations are not limited by specific storage through read and write operations. An added lock region is used for inter-node competition for lock resources. Disk regions are partitioned to target metadata structures requiring concurrent access protection. Then, based on the Paxos algorithm and the kernel block layer interface, I / O requests are directly issued to allow target nodes to compete for lock resources. This avoids the problem of existing clustered file systems being unable to simultaneously achieve network independence and storage independence during concurrent access, improving the stability and usability of clustered files.

A telemetry processing system in a cluster network generates telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. Producers can define new metrics for use in a running network. Schema for a new metric is validated by a schema validator and stored in a telemetry catalog. Telemetry datasets for the new metric are sent from the producer for storage in a datastore. Consumers are notified as to the availability of the new metric, and those who elect to receive the new metric are sent the new datasets along with their previous datasets.

A telemetry processing system in a cluster network generates telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. Users of the telemetry data are mapped to specific role-based access control (RBAC) rules per an identity and management (LAM) module. This mapping is stored in a dynamic RBAC-based telemetry catalog for further rules checking as telemetry data is generated. This adds a second layer above user subscription terms to safeguard the security of telemetry data based on RBAC rules, and allows the system to define conditions under which certain users can receive certain types of telemetry data in an efficient and dynamic manner.

Heatmaps are used to identify access patterns of a series of data blocks at specific times to optimize the size of the replication node set for the purpose of reducing access latency. In one embodiment, heatmaps are used to enforce minimal replication for each data block, such as when a replication node crashes and is replaced. In another embodiment, heatmaps are used to adjust the minimum replication of data blocks during and after peak demand periods. In one embodiment, each data block is replicated on the minimum number of corresponding replication nodes in the cluster. A first node requests access to a data block. Based on the requested access, the heatmap is modified, and the data block is replicated to the first node. Based on the heatmap, the minimum number of nodes in the corresponding replication nodes for at least one data block is adjusted.

The invention discloses a file request processing method and electronic equipment, relates to the technical field of storage systems, and comprises the step of simplifying lock management in a cluster file system by routing a file request to a corresponding file home node. When a file request is received, the index node identifier is analyzed according to the file path, so that the distribution group where the file is located is determined. And then mapping the distribution group to a corresponding file affiliation node by using a global view, and forwarding the request to the node for processing. Due to the fact that all operations aiming at the same file are routed to the same home node, the node can use a simple local lock mechanism to guarantee data consistency, complex distributed lock coordination is not needed, lock management logic is simplified, meanwhile, network communication overhead is reduced, and the overall performance and reliability of the system are improved.

A telemetry processing system in a cluster network generates telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. The system automatically verifies a datatype of telemetry data generated by a pod as a telemetry producer as corresponding to processed data requiring compliance with a defined regulation. Appropriate compliance rules are loaded in a checklist of a telemetry transmitter, and then installed in a compliance library maintained in each pod. As it is generated, the telemetry data is checked against a respective compliance library, and transmitted to a datastore for storage and collection by a telemetry collector of a telemetry transmitter.

Updating fingerprints for deduplicated data segments in a cluster network by calculating new fingerprint hash values using a new hash process. A new index mapping from the present hash process to the new hash process is created, as is a new fingerprint index mapping from the updated hash process to the containers. The process generates new fingerprints with the new fingerprint hash, and calculates the new fingerprint hash values of the new data segments using the new hash process. Relevant data structures are updated using the current fingerprint hash values with the new fingerprint hash values. New containers are created as needed to store the new fingerprint hash values. Old data elements like containers and fingerprints are deleted during the upgrading process to reduce space overheads.

The invention provides a high-availability database system based on a cluster file system and a read-write method, and the database system comprises a database client which is used for receiving an access request and sending the access request to a database server; the database server is used for processing the access request sent by the database client; the number of nodes included in the database server is an odd number, and the database server comprises real nodes and virtual nodes; and the cluster file system is used for processing the request sent by the database server. According to the invention, two physical servers can be deployed, one database record is stored in each physical server, one database record is stored in the cluster file system, and two data can be available when any physical server fails, so that the normal operation of services is ensured. According to the method, the number of servers required for ensuring high availability of the cluster is reduced from at least three to two, so that the hardware investment of an enterprise is saved, and the cost is reduced.

A multi-node, multi-container cluster system that generates, aggregates, and manages log files from services and components to be used for audit logs used by an auditor. Logs are collected from all components of the system and aggregated using a central persistent volume for non-critical applications and additional dedicated persistent volumes for critical tasks. Audit logs are formatted to provide a consistent logging format in conformance with relevant corporate and government requirements. Log files are stored persistently outside of pods to ensure high availability for any audit log management needs. Audit logs can be exported to external, secure syslog server, and can be made subject to defined retention policies, as needed.

A computer-implemented method, according to one approach, includes: receiving a request to create a new node in a clustered filesystem. In response to receiving the request, a copy of details is retrieved, the copy of details being associated with an established singleton cluster preconfigured to access the clustered filesystem. The retrieved copy of details are further applied to a new node in the clustered filesystem. Moreover, data structures of the new node are modified according to the received request. Other systems, methods, and computer program products are described in additional approaches herein.

The embodiment of the invention provides a directory entry processing method and device, a storage medium and a program product, and relates to the field of distribution. The method comprises the following steps: receiving an operation for creating a directory or newly adding a directory entry; the directory entry count is updated in response to the operation, the directory entry count is used for recording the total number of files and subdirectories in a target directory, and the target directory is a directory targeted by the operation; in a case where the value of the directory entry count satisfies a large directory condition, a continuous space including N blocks is allocated in the resource group, and a new directory entry generated based on the operation is stored in the continuous space, where N is greater than 1. According to the method provided by the invention, the continuity of metadata access is increased, and the traversal and statistical performance of the cluster file system during processing of massive file directories is improved.

A telemetry processing system in a cluster network generates telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. Producers can define new metrics for use in a running network. Schema for a new metric is validated by a schema validator and stored in a telemetry catalog. Telemetry datasets for the new metric are sent from the producer for storage in a datastore. Consumers are notified as to the availability of the new metric, and those who elect to receive the new metric are sent the new datasets along with their previous datasets.