216 results about "Metadata service" patented technology

A framework for implementing multitenant architecture is provided. The framework comprises a framework services module which is configured to provide framework services that facilitate abstraction of Software-as-a-Service (SaaS) services and crosscutting services for a Greenfield application and a non SaaS based web application. Further the abstraction results in a SaaS based multitenant web application. The framework further comprises a runtime module configured to automatically integrate and consume the framework services and APIs to facilitate monitoring and controlling of features associated with the SaaS based multitenant web application. The framework further comprises a metadata services module configured to provide a plurality of metadata services to facilitate abstraction of storage structure of metadata associated with the framework and act as APIs for managing the metadata. The framework further comprises a role based administration module that facilitates management of the metadata through a tenant administrator and a product administrator.

According to one embodiment, a storage system (400) may include a gateway service (402), metadata service (404) and storage service (406) that each provide different storage system functions. Each service (402, 404 and 406) may include a plurality of servers that are functionally de-coupled from one another. De-coupled servers may service requests without regard to the operation of other servers of the same service. Such a de-coupled arrangement can allow a storage system (400) to be scaled to meet increasing demands, as servers may be added to a system, on a service by service basis, without necessarily re-indexing stored information or reconfiguring multiple instances of a server application. Further, backup servers may be easily added to the system to enable fault tolerance. Servers of a particular service (402, 404 and 406) may be accessed according to a quality of service policy.

The invention relates to a method and a system for load balancing of metadata management in a distributed file system. When the data structure of metadata, i.e. a directory tree, becomes hotspot metadata, the access heat degree of the directory tree is updated through adopting a mode of copying the metadata; when the load balancing degrees of service nodes of the metadata exceeds a preset value, partial working loads are transferred to underload nodes from overload nodes through adopting a mode of transferring the metadata; and the loading balancing among a plurality of the metadata service nodes is realized through the two modes. The system comprises an information communication module, a metadata operation module, a metadata service node management module, a metadata management module, a metadata strategy implementation nodule and a working load acquisition module. The invention improves the utilization rate of metadata service resources through the two modes of copying and transferring the metadata so as to achieve the purpose of load balancing, thus a metadata service system can be smoothly expanded.

The present invention extends to methods, systems, and computer program products for creating proxies from service description metadata at runtime. A proxy creation request is received from a requesting application while the requesting application is executing at runtime. The proxy creation request requests creation of a proxy instance to facilitate compatible communication between the requesting application and another application. A proxy instance is created in response to the proxy creation request. Service description metadata describing the other application is accessed (e.g., through a request to a metadata service or retrieval from cache). The proxy instance is configured in accordance with the service description metadata describing the other application. Accordingly, the proxy instance can facilitate compatible communication between the requesting application and the other application to implement operations that the requesting application requests to be performed at the other application.

A method for improving metadata service reliability and a relative system belongs to computer memory technical field, for fully utilizing node resource to improve the reliability of metadata service. The inventive method comprises normal operation, main metadata server accident, backup metadata server lift, sub metadata server accident and second backup metadata server lift. The inventive system comprises a client node, a memory node, a main metadata server and a sub metadata server, which respectively comprises an object metadata service module, a memory resource manage module, and a high available module, wherein the high available module comprises a metadata copy module, an accident shield module, a detect module and a select module, the select module can select the memory node with minimum load as the backup metadata server, and the accident shield server is switched to next metadata server, and selects the backup metadata server to form a master-salve backup chain service mode, to improve the reliability of metadata service.

The invention relates to the technical field of computer application, in particular to a cloud database system used for distributed heterogeneous data resource integration. The cloud database system is composed of a metadata service node, a service node, a front-end computer node and a production computer node, the metadata service node provides a system service to realize overall management of the whole cloud database system, the service node mainly provides a data service, various heterogeneously-distributed databases or file systems on the front-end computer node serve as data sources of the cloud database system, and the data sources are accessed to the inside of the cloud database system through an access mechanism provided by the cloud database system; a production system writes data into a database system on the production computer node in real time, and then the data are periodically copied on a front-end computer by a data synchronization module of the cloud database system. The cloud database system solves the problem of integration of distributed heterogeneous information resources and can be applied in a distributed-type heterogeneous database.

Aspects of the present disclosure are directed to methods and systems for protecting sensitive data in a hosted service system. The system includes a host system and the host system includes a key management system (KMS) and a metadata service system (MSS). The KMS and the MSS are communicatively coupled to each other. The system further includes a database management system (DBMS) having a database, a query pre-parser, and a results handler. The query pre-parser and the results handler are communicatively coupled to the KMS and the MSS, and the system also includes a processing application adapted to process at least some data received from a tenant system.

A method and system for collecting metadata for multimedia objects is described. According to an exemplary embodiment, a metadata service is provided for a multimedia object, wherein the metadata service is not associated with a source of the object. At least one metadata vocabulary is identified for the multimedia object using the metadata service. Metadata related to the multimedia object is collected according to the at least one metadata vocabulary. The metadata service allows users to view, modify, or search the metadata for the multimedia object. Users can further select or create metadata vocabularies to be used in the viewing, modifying, or searching of the metadata. In this manner, a common mechanism for providing a metadata service related to the multimedia object and for advertising the metadata service is provided.

A system and method are provided for deduplication (dedup) of data file information in a network of distributed data filers. A host, including a metadata service (MDS) and a data node (DN), receives a block count and hash group calculations for a file from a network-connected client device. The MDS creates a file map with target addresses, and compares the calculated hash group to hash groups associated with stored blocks of data in a global dedup hash table. If a match is found, the MDS directs the client device not to send the block of data associated with matching hash group, and reallocates the address of the block of data from the target address.

The invention relates to a distributed block storage system and a data routing method thereof. The storage system comprises a block storage access client, a storage server cluster and a metadata service cluster that are connected through a storage network separately, wherein the block storage access client is used for receiving a read write request of an upper layer application and converting theread write request of the upper layer application into a parallel read write request for storage objects of different copies, the storage server cluster comprises multiple storage service nodes that are used for storing data and responding to the parallel read write request, and the metadata service cluster is used for storing cluster configuration information of the storage server cluster and monitoring a real-time state of the storage server cluster. In the distributed block storage system, data distribution is based on a distributed hash algorithm, and all data objects of storage volumes are uniformly distributed in all storage service nodes. Compared with the prior art, the distributed block storage system has advantages of shortening read write request link transmission delay and improving distributed storage system performance.

This invention discloses a coexistence structure of a multi-use unit of a parallel storage system including a state test and control frame, a data service frame, a metadata service frame, a data synchronous frame, a customer end frame, a system management frame and a high usability module supporting load and unload of on-line modules, in which, the high usability module realizes that all frames call necessary interface functions in the mode of plug-in components and the frames call corresponding interfaces realized in the usable module based on the high usability kind used by logic data to finish specific function, and users can select the most suitable unit in the high usability unit to secure reliability and usability of logic data.

The invention discloses a big data storage and access system and method based on a distributed file system. The big data storage and access system based on the distributed file system comprises the distributed file system which comprises a metadata server, a storage server cluster, an access client and a log server, the metadata server manages and monitors metadata information in the distributed file system and the storage server cluster, the storage server cluster stores data contained in the metadata information in multiple data storage nodes of the storage server cluster, the log server backs up the metadata information in the metadata server and synchronizes operation logs of the metadata server in real time, and the access client is used for selecting standby metadata service processes in the metadata server or in the log server according to the connection state of the metadata server to store the server cluster.

The invention provides a method and a device of metadata processing. The metadata processing method includes steps of: a. receiving write requests with object identity, b. generating at least one modify instruction corresponding to the object identity according to the write requests, wherein the modify instruction includes a data identity and a target value corresponding to the data identity, and the data identity is at least one metadata identity needing modifying of the metadata corresponding to the target identity, c. sending the modify instruction to a client terminal and a data server which are corresponding to the data identify, so that the client terminal and the data server respectively modify the metadata corresponding to the data identify into target value according to the modify instrument. Therefore, metadata stored in the client terminal and the data server are updated synchronously, storing uniformity of data in a distributed system is guaranteed, write conflict caused by the fact that multiple client terminals send the write requests at the same time is avoided, and performance of providing metadata service of the distributed system is guaranteed effectively.

A distributed storage construction method based on mimicry construction comprises the steps that a storage driving module receives user request information, encapsulates and converts the user requestinformation and then sends the user request information to a dispatcher; the dispatcher receives the user request information, preprocesses the user request information, and sends the user request information to a plurality of heterogeneous metadata service executors in an active state at the same time according to a distribution rule; the heterogeneous metadata service executor in the active state receives the user request information forwarded by the dispatcher, carries out corresponding request processing operation and sends a processing result to the adjustor; the adjustor receives the processing result information sent by each heterogeneous metadata service executor, classifies the processing result information according to the request sources, performs consistency voting on multiplepieces of processing result information of the same request source, and sends the voted result information to the storage driving module after proper processing; and the storage driving module returnsuser result information to the user after receiving the result information generated after voting of the adjustor.

A method for automatically synchronizing multi-source heterogeneous data resources utilizes a metadata service node, a service node and a production node, wherein the metadata service node stores a data dictionary of a distributed heterogeneous database system, the service node parses user requests of the distributed heterogeneous database system, for regularly synchronizing data with the production node; and the production node provides requests such as querying and updating of a relevant production application system; wherein the service node includes a configuration analyzer, a log parser and a synchronous update device; the configuration analyzer analyzes source database configurations on the production node, and selects an applicable synchronization method; the log parser parses logs of different database systems, which is a preparatory component of incremental synchronization; the synchronous update device recognizes the synchronization method according to the configuration analyzer for data synchronization.

The invention discloses a high-efficiency data access method of a cloud storage client. The method comprises the following steps: A, allowing the client to generate a file access request according to user operation, B, inquiring whether metadata of a file to be accessed by the file access request has corresponding cache locally in the client, executing Step D if so, and executing Step C if not, C, allowing the client to upload and download the metadata from a metadata service node of a cloud service terminal, updating the local metadata cache of the client, and finishing the process, and D, reading corresponding metadata information from the local metadata cache of the client. The invention further discloses a cloud storage client. The method and the client can improve access capability of a cloud storage system, and solve the defects of low response speed of accessing to data of the cloud storage service terminal, low concurrence efficiency and low bandwidth utilization ratio.

The invention discloses a metadata service system metadata synchronized method and writing server updating method. The metadata service system provided by the invention includes a writing server and a reading server; the writing server is used for storing the metadata in a concurrent file system to receive reading access and receive writing access; correcting the metadata in the concurrent file system and synchronously updating the corrected metadata into the reading server; the reading server is used for storing the metadata and receiving reading access, receiving the synchronous updating of the writing server to the metadata, and is also used for converting into writing service when the write server is monitored to be valid. The invention solves the problem of single point failure through the mutual backup of the reading server and the writing server, and can meet high effective large-scale concurrent access demands.

The invention provides a metadata management method. The metadata management method comprises the following steps: acquiring metadata position information from a metadata position service module; transmitting the metadata position information to a metadata service module; performing reading and writing operation on metadata according to the metadata position information. By the metadata management method, metadata management is divided into a metadata position management layer and a metadata management layer, the metadata position information accessed by users is uniformly managed through the metadata position management layer, and the metadata management layer disperses access requests of the users to a plurality of servers to execute according to the metadata position information, so that the defects of long system response time, slow response, low access efficiency and the like caused by simultaneous access to the metadata storage system by multiple users can be overcome, access pressure caused by simultaneous access to the metadata storage system by multiple users can be handled, and the system performance is improved.

The invention discloses a distributed metadata management method and system, and relates to the technical field of metadata management. By means of the method and the system, two strategies including static load balancing when metadata is distributed and dynamic load balancing in system operation are used; therefore, the utilization rate of metadata service resources is increased; the load balancing of the system is ensured; the expandability of the system is increased; in addition, because a metadata delay movement scheme based on a directory redirection table is used, the problem of a large number of metadata movement due to renaming operation can be solved; and the stability of the system efficiency is ensured.

The invention relates to a method for implementing distributed file systems on the basis of hierarchical storage. The method includes steps of 1), establishing the distributed file systems without metadata service for data in systems by the aid of global unified name spaces; 2), dividing the integral distributed file systems into different logical partitions according to business requirements; 3), selectively utilizing different storage media and storage modes for the different logical partitions; 4), automatically migrating data of the logical partitions and repositioning the migrated data. Compared with the prior art, the method has the advantages of complete data access parallelization, good performance, low hardware cost and the like.