37 results about "Clustered file system" patented technology

A telemetry processing system in a cluster network generates telemetry data from a plurality of telemetry producers and formats it into a structured format for storage. Producers define new metrics for use in a running network. Role-based access control (RBAC) levels for metrics are mapped to users based on roles within an organization using the produced telemetry data. A final RBAC level of a new metric is established through a recommendation and validation process between the producer and an RBAC recommendation module. Once validated, the final RBAC level and new metric are stored in a telemetry catalog. Telemetry datasets for the new metric are sent for storage and transmitted to appropriate users based on RBAC rules including the final RBAC level.

A telemetry processing system in a cluster network receives telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. A telemetry producer can identify consumers to receive a new metric dataset as part of the telemetry data. This data may include sensitive data subject to compliance regulations with respect to data security and privacy. A telemetry collector validates schema of the new metric dataset and a list of consumers through an acceptance or rejection decision. If accepted, the listed consumers can receive the new metric dataset, otherwise they are blocked. This prevents the exposure of sensitive data to unvetted consumers.

The invention provides a cluster file system of a database. Wherein the cluster file system of the database comprises an application layer, a proxy distribution layer, a hash ring layer, a file system layer and a shared storage layer, operation on object data can be distributed through the proxy distribution layer, the same object data is only processed at a certain physical node, lock resources are prevented from being scrambled, and the operation efficiency is improved. Meanwhile, each physical node caches the corresponding partition resource group lock into the memory of each physical node, and lock switching is not carried out when needed, so that the time delay of lock switching is reduced; therefore, each physical node caches the metadata of the object data of the corresponding partition in the memory of each physical node, the metadata is not obtained from a disk when needed, and the time delay of disk IO is reduced.

File names and inodes are indexed directly in a filesystem namespace consisting of a layered set of key value store (KVS) database logical tables. The KVS is used directly for organizing file and directory metadata on a cluster file system instead of traditional Btree structures. Embodiments provide file deletion in the file system namespace, and other features include snapshot deletion and cleaning the namespace. A name hash index is used to speed filename lookup in the file system namespace. Two key types of INODE and DIRENT are used to encode namespace metadata using self-identify variable length records in the namespace. Point-in-time snapshot copies are also formalized using KVS database operations as well operations on sets of snapshots for operations including garbage collection and data replication.

A system can, for a cluster that comprises a group of nodes and that hosts a group of virtual machines, create and attach respective new virtual network interface cards to respective virtual machines of the group of virtual machines, wherein the respective new virtual network interface cards are assigned to the new external subnet. The system can create the new external subnet. The system can delete respective current virtual network interface cards from the respective virtual machines, wherein the respective current virtual network interface cards are assigned to the current external subnet. The system can acquire respective dynamic host control protocol leases for the respective virtual machines, wherein the respective dynamic host control protocol leases correspond to the new external subnet. The system can halt a connection from a computer on the new external subnet via the respective new virtual network interface cards.

A key value store is maintained in which the key value store includes pages of a B+ Tree structure holding file metadata of a logical partition of a filesystem. The pages are associated with keys through which the pages are retrieved from the key value store. The keys are named. A clone is created of the logical partition, the logical partition now being a parent and the clone now being a child. The naming of the keys is used to allow the pages stored in the key value store to be read by any of the parent or child. The naming of the keys is used to allow any of the parent or child to make updates to the pages, where updated pages are owned by a respective parent or child that made the updates, and pages not updated remain accessible by any of the parent or child.

A computer-implemented method for provisioning cloud computing clusters includes receiving a request to create a cloud computing cluster, the cloud computing cluster comprising a clustered filesystem and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster and initiating an initialization process for a single processing node and a corresponding attached storage device responsive to receiving the request to create the cloud computing cluster. The method may also include, previous to completion of the initialization process, requesting and receiving an IP address for the single processing node and a device ID for the corresponding attached storage device from one or more cloud infrastructure controllers and configuring the clustered filesystem and a corresponding WAN cache using the received IP address and the received device ID. A system and computer program product corresponding to the above method are also disclosed herein.

Forechannel connections are established from client-side deduplication libraries to a namespace node of a set of namespace nodes formed as a cluster. The namespace nodes are responsible for namespace operations on files managed by a deduplication filesystem. The forechannel connections are responsible for carrying file data between the client-side deduplication libraries and the namespace node. Server-initiated connections are established from the namespace node to the client-side deduplication libraries. The server-initiated connections are responsible for carrying instructions from the namespace node to the client-side deduplication libraries. While a transfer of file data is in progress, the client-side deduplication libraries are instructed over the server-initiated connections to failover the transfer to one or more other namespace nodes. After the failover, the namespace node is upgraded.

A telemetry processing system in a cluster network receives telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. One or more consumers subscribe to receive respective data of the telemetry data. A selected transport interface transmits the appropriate telemetry datasets to subscribed consumers. The system automatically updates the subscriptions of the consumers to conform any update to the telemetry data, the telemetry producers, or transport mechanisms.

This invention discloses a method, apparatus, and medium for accessing a clustered file system, applicable to virtualization and hyper-converged infrastructure. Based on block file data I / O operations, the Paxos algorithm is applied to determine the target node for lock request access. The entire process achieves node-to-node access without relying on the network. I / O operations are not limited by specific storage through read and write operations. An added lock region is used for inter-node competition for lock resources. Disk regions are partitioned to target metadata structures requiring concurrent access protection. Then, based on the Paxos algorithm and the kernel block layer interface, I / O requests are directly issued to allow target nodes to compete for lock resources. This avoids the problem of existing clustered file systems being unable to simultaneously achieve network independence and storage independence during concurrent access, improving the stability and usability of clustered files.

A telemetry processing system in a cluster network generates telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. Producers can define new metrics for use in a running network. Schema for a new metric is validated by a schema validator and stored in a telemetry catalog. Telemetry datasets for the new metric are sent from the producer for storage in a datastore. Consumers are notified as to the availability of the new metric, and those who elect to receive the new metric are sent the new datasets along with their previous datasets.

A telemetry processing system in a cluster network generates telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. Users of the telemetry data are mapped to specific role-based access control (RBAC) rules per an identity and management (LAM) module. This mapping is stored in a dynamic RBAC-based telemetry catalog for further rules checking as telemetry data is generated. This adds a second layer above user subscription terms to safeguard the security of telemetry data based on RBAC rules, and allows the system to define conditions under which certain users can receive certain types of telemetry data in an efficient and dynamic manner.

Heatmaps are used to identify access patterns of a series of data blocks at specific times to optimize the size of the replication node set for the purpose of reducing access latency. In one embodiment, heatmaps are used to enforce minimal replication for each data block, such as when a replication node crashes and is replaced. In another embodiment, heatmaps are used to adjust the minimum replication of data blocks during and after peak demand periods. In one embodiment, each data block is replicated on the minimum number of corresponding replication nodes in the cluster. A first node requests access to a data block. Based on the requested access, the heatmap is modified, and the data block is replicated to the first node. Based on the heatmap, the minimum number of nodes in the corresponding replication nodes for at least one data block is adjusted.

A telemetry processing system in a cluster network generates telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. The system automatically verifies a datatype of telemetry data generated by a pod as a telemetry producer as corresponding to processed data requiring compliance with a defined regulation. Appropriate compliance rules are loaded in a checklist of a telemetry transmitter, and then installed in a compliance library maintained in each pod. As it is generated, the telemetry data is checked against a respective compliance library, and transmitted to a datastore for storage and collection by a telemetry collector of a telemetry transmitter.

Updating fingerprints for deduplicated data segments in a cluster network by calculating new fingerprint hash values using a new hash process. A new index mapping from the present hash process to the new hash process is created, as is a new fingerprint index mapping from the updated hash process to the containers. The process generates new fingerprints with the new fingerprint hash, and calculates the new fingerprint hash values of the new data segments using the new hash process. Relevant data structures are updated using the current fingerprint hash values with the new fingerprint hash values. New containers are created as needed to store the new fingerprint hash values. Old data elements like containers and fingerprints are deleted during the upgrading process to reduce space overheads.

The invention provides a high-availability database system based on a cluster file system and a read-write method, and the database system comprises a database client which is used for receiving an access request and sending the access request to a database server; the database server is used for processing the access request sent by the database client; the number of nodes included in the database server is an odd number, and the database server comprises real nodes and virtual nodes; and the cluster file system is used for processing the request sent by the database server. According to the invention, two physical servers can be deployed, one database record is stored in each physical server, one database record is stored in the cluster file system, and two data can be available when any physical server fails, so that the normal operation of services is ensured. According to the method, the number of servers required for ensuring high availability of the cluster is reduced from at least three to two, so that the hardware investment of an enterprise is saved, and the cost is reduced.

A computer-implemented method, according to one approach, includes: receiving a request to create a new node in a clustered filesystem. In response to receiving the request, a copy of details is retrieved, the copy of details being associated with an established singleton cluster preconfigured to access the clustered filesystem. The retrieved copy of details are further applied to a new node in the clustered filesystem. Moreover, data structures of the new node are modified according to the received request. Other systems, methods, and computer program products are described in additional approaches herein.

The embodiment of the invention provides a directory entry processing method and device, a storage medium and a program product, and relates to the field of distribution. The method comprises the following steps: receiving an operation for creating a directory or newly adding a directory entry; the directory entry count is updated in response to the operation, the directory entry count is used for recording the total number of files and subdirectories in a target directory, and the target directory is a directory targeted by the operation; in a case where the value of the directory entry count satisfies a large directory condition, a continuous space including N blocks is allocated in the resource group, and a new directory entry generated based on the operation is stored in the continuous space, where N is greater than 1. According to the method provided by the invention, the continuity of metadata access is increased, and the traversal and statistical performance of the cluster file system during processing of massive file directories is improved.

A telemetry processing system in a cluster network generates telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. Producers can define new metrics for use in a running network. Schema for a new metric is validated by a schema validator and stored in a telemetry catalog. Telemetry datasets for the new metric are sent from the producer for storage in a datastore. Consumers are notified as to the availability of the new metric, and those who elect to receive the new metric are sent the new datasets along with their previous datasets.

A telemetry processing system in a cluster network receives telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. An original telemetry producer can identify other producers to transmit a new metric dataset for transmission to registered consumers. A schema validator module validates schema of the new metric dataset and a list of the other producers through an acceptance or rejection decision. If accepted, the other producers can store and transmit the new metric dataset, otherwise they are blocked. This prevents the generation of superfluous telemetry data in the network by an excessive number of producers.

The invention relates to a fault processing method of a cluster file system and a related product. The fault processing method for the cluster file system comprises the following steps: in response to an operation request of a user on resources in the cluster file system, judging whether the cluster file system is in a fault log recovery state or not; and under the condition that the cluster file system is in the fault log recovery state, executing locking operation on the resources. And judging whether the main node of the lock is a fault node. And under the condition that the main node of the lock is the fault node, searching records about resources in the log, and playing back the records about the resources in the log by using a log playback thread pre-started by the fault node. And executing the operation according to the operation request after the pay-back thread is completed. According to the fault processing method of the cluster file system, log playback is not put in a node fault processing flow, so that the node fault processing time can be effectively shortened, and the influence of node faults on other normal services can be reduced.

A telemetry processing system in a Kubernetes-based cluster network receives golden signal telemetry data from a plurality of pods and formats it into a structured format for validation and storage in a datastore. The golden signals include are generated during data protection operations and are used to reveal problems encountered by the pods. A latency golden signal measures a time between operation request and initiation, the traffic golden signal measures an amount of activity present in an initiated operation, the errors golden signal measures a rate at which requests to the backup server are failing, and the saturation golden signal measures system utilization in terms of how much excess capacity the initiated operation has available.

The invention provides a cluster file system of a database. The cluster file system of the database comprises a shared storage module, a distributed lock management module, a striped metadata layout module, a distributed block allocation strategy module and a lock granularity optimization module. The striped metadata layout module is configured to generate striped metadata in the creation stage of the file and persist the striped metadata to inode metadata of the file, and the striped metadata records the total number of stripes of the file and the block initial address of each stripe; the distributed block allocation strategy module is configured to allocate each strip to a predefined block group or block area in the shared storage module according to the strip number of the file, so as to complete the block allocation of the data to be allocated; and the lock granularity optimization module is configured to construct an independent strip sub-lock for each strip on the basis of the inode file lock of the distributed lock management module, each strip sub-lock independently controls a block area covered by the strip sub-lock, and parallel access of multiple nodes or multiple processes to different strips of the same file is realized.

A system can maintain a computer cluster that comprises a group of nodes, wherein a node of the group of nodes comprises a group of storage drives, wherein the node is a member of a failure domain that comprises a subgroup of nodes of the group of nodes, wherein the failure domain is configured to preserve data stored in the failure domain when at least one node within the failure domain fails. The system can obtain a reservation for the node, wherein the reservation permits the node to make the group of storage drives unavailable for data access, and wherein other nodes within the failure domain are unable to obtain the reservation while the node possesses the reservation. The system can, while the node possesses the reservation, update firmware for respective storage drives for the group of storage drives in parallel.

The application discloses a data processing method of a cluster file system and electronic equipment, and relates to the technical field of computers, which comprises the following steps: receiving a data processing request, the data processing request being processed by a first node in a plurality of nodes of a cluster file system, the plurality of nodes being used for managing a plurality of storage spaces in the cluster file system; determining a first identifier in the data processing request and a type of the data processing request; determining a first thread in a plurality of thread pools of the first node according to the type of the data processing request and the first identifier; and processing the data processing request according to the first thread. In the method, the plurality of nodes are used for managing the plurality of storage spaces in the cluster file system, the data processing request can be dispersed to each node for independent processing, communication delay caused by frequent contention for global locks and other resources among the nodes can be reduced, and therefore the response speed of the cluster file system is accelerated.

A telemetry processing system in a cluster network generates telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. Producers can define new metrics for use in a running network. Schema for a new metric is validated by a schema validator and stored in a telemetry catalog. For a metric that is to be enabled for only a short duration, the producer registers the metric with a duration and condition for triggering collection of metric datasets. A telemetry collector detects the condition and collects the datasets from the producer and stops collection at the end of the specified duration. Any short-term metric that is no longer needed or requires updating can be unregistered to clean up the telemetry catalog.

A multi-node, multi-container cluster system that generates, aggregates, and manages log files from services and components to be used for audit logs used by an auditor. Logs are collected from all components of the system and aggregated using a central persistent volume for non-critical applications and additional dedicated persistent volumes for critical tasks. Audit logs are formatted to provide a consistent logging format in conformance with relevant corporate and government requirements. Log files are stored persistently outside of pods to ensure high availability for any audit log management needs. Audit logs can be exported to external, secure syslog server, and can be made subject to defined retention policies, as needed.

The invention provides a file searching method and device of a cluster file system, a medium and a product. The file searching method of the cluster file system comprises the steps of obtaining a file path of a target file; obtaining the name of a path node of each level in the file path and the level of each path node in the directory tree; based on the name of each path node and the hierarchy where the path node is located in the directory tree, corresponding metadata information is searched in hierarchical caches corresponding to the path nodes in parallel, the cluster file system is configured with a plurality of hierarchical caches, and all the hierarchical caches correspond to all the hierarchies of the directory tree one to one; each hierarchy cache stores metadata information of all nodes of one hierarchy of the directory tree; and returning metadata information of the target file according to search results of all the path nodes. Parallel searching of the file paths is achieved, dependence of a traditional query process on the sequence of the file paths is eliminated, and the file searching efficiency under the directory tree, especially under a large-scale directory tree, can be improved.

A telemetry processing system in a cluster network generates telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. Users of the telemetry data are mapped to specific role-based access control (RBAC) rules per an identity and management (LAM) module. This mapping is stored in a dynamic RBAC-based telemetry catalog for further rules checking as telemetry data is generated. This adds a second layer above user subscription terms to safeguard the security of telemetry data based on RBAC rules, and allows the system to define conditions under which certain users can receive certain types of telemetry data in an efficient and dynamic manner.

A telemetry processing system in a cluster network receives telemetry data from a plurality of telemetry producers and formats it into a structured format for storage in a datastore. One or more consumers subscribe to receive respective data of the telemetry data at a particular reception frequency, such as once per day or week, etc. A selected transport interface transmits the appropriate telemetry datasets to subscribed consumers according to their selected schedule to allow telemetry producer to collect telemetry data in tune with consumer schedules.