838 results about "Metadata server" patented technology

For enhanced access control, a client includes a token in each read or write command sent to a block storage device. The block storage device evaluates the token to determine whether or not read or write access is permitted at a specified logical block address. For example, the token is included in the logical block address field of a SCSI read or write command. The client may compute the token as a function of the logical block address of a data block to be accessed, or a metadata server may include the token in each block address of each extent reported to the client in response to a metadata request. For enhanced security, the token also is a function of a client identifier, a logical unit number, and access rights of the client to a particular extent of file system data blocks.

A reference to auxiliary data is steganographically encoded within a media signal. The auxiliary data is stored in a metadata database that maps the reference encoded in the media signal to its corresponding metadata. Various application programs and devices can access the metadata by extracting the reference from the media signal, and querying the database for metadata corresponding to the reference. The metadata database may be implemented in a network server to make metadata readily available to devices and applications distributed throughout a network. This steganographic link to metadata may be used to retrieve metadata across media management systems. In one configuration, for example, media management systems have metadata servers that maintain metadata for a collection of media signals. The metadata server is responsible for responding to requests for metadata associated with media titles in its collection. In the event that a metadata server does not have metadata associated with a particular media title, it forwards the reference extracted from the media signal to a metadata router, which in turn, re-directs the request to the appropriate metadata server.

A system is described in which a media browser (101), operating as a software application on a user terminal or preferably a server for a number of users, provides a user with a single user interface that facilitates browsing and searching different metadata collections over the Internet (102). A metadata server (212) is associated with each of the metadata collections. When the metadata server (212) receives a request from the media browser (101), the metadata server (212) interprets the request and replies with a description that satisfies the request and according to a predetermined scheme. The description contains at least one link which represents a return link which represents a return request to the metadata server (212). Specifically disclosed are methods (2000) for recording user location preferences at a service and using one or more lists of such preferences to focus searching responses to statistically desirable material. Monitoring the validity of a bookmarked location is also described.

A system and method for building virtual appliances using a repository metadata server and a dependency resolution service is provided. In particular, a hosted web service may provide a collaborative environment for managing origin repositories and software dependencies, whereby remote clients may follow a simple and repeatable process for creating virtual appliances. For example, the repository metadata server may cache and parse metadata associated with an origin repository, download software from the origin repository, and generate resolution data that can be used by the dependency resolution service. The dependency resolution service may then use the resolution data to resolve dependencies for a package selected for an appliance, wherein the dependencies may include packages that are required, recommended, suggested, banned, or otherwise a dependency for the selected package.

A system is described in which a media browser (101), operating as a software application on a user terminal or preferably a server for a number of users, provides a user with a single user interface that facilitates browsing and searching different metadata collections over the Internet (102). A metadata server (212) is associated with each of the metadata collections. When the metadata server (212) receives a request from the media browser (101), the metadata server (212) interprets the request and replies with a description that satisfies the request and according to a predetermined scheme. The description contains at least one link which represents a return link which represents a return request to the metadata server (212). Specifically disclosed are methods of forming a table of contents for a particular user preferably based upon media reproduction attributes, methods of right to use and quality of service control, graphical user interfaces for facilitating the browsing process, methods of interpreting metadata, methods of communicating metadata between users, and method for transferring media sessions between users.

The invention relates to the technical field of Internet information, in particular to a cloud storage system and an implementation method thereof. The cloud storage system comprises more than one metadata server sets and a plurality of data block storage node sets, wherein metadata server sets manage the whole distributed file system, respond an access of a client to a file, receive a writing request of a writing client and distribute the writing request to a storage block or receive a request of a reading client for reading a file and return to a position of a block corresponding to the file; and the reading client reads file blocks from corresponding blocks and combines the file blocks. In a system, the file is divided into one or more blocks; the blocks are scatteredly stored in the data block storage node sets; and storage nodes create, delete and copy the file blocks under the command of metadata servers. In the cloud storage system and the implementation method, the safety and the transmission efficiency are improved due to block redundancy distribution and block balance distribution, and the congestion probability is reduced. The cloud storage system and the implementation method are widely applied to cloud storage systems of enterprises.

An apparatus includes a computing node and a metadata server. The computing node may transmit a request to open a file to the metadata server, which may provide a corresponding file identifier to the computing node. The computing node may use the file identifier to directly access the storage storing the file. In one embodiment, the storage may be an object-based storage. The storage may receive the file identifier with an access command and may perform the mapping from file identifier to storage blocks internally. Thus, accesses to the storage may be performed on an arbitrary boundary within the file. In other words, the entire block including the data needed by the computing node may not be transferred to the computing node. Instead, the storage may select the data requested from the file (e.g. via an offset and a number of bytes or some similar mechanism) and return that data to the computing node.

The invention discloses a distributed file system metadata management method facing to high-performance calculation. The method comprises the following steps of: 1) establishing a catalogue metadata server cluster, a file metadata server cluster and an object storage server cluster; 2) establishing a global counting-based bloom filter in the catalogue metadata server cluster; 3) when the operation request of a client side arrives, skipping to execute step 4) or 5); 4) enabling the catalogue metadata server cluster to respond to the catalogue operation request of the client side to manage the catalogue metadata; and 5) enabling the file metadata server cluster to respond to the file operation request of the client side to manage the file metadata data. According to the distributed file system metadata management method disclosed by the invention, the metadata transferring problem brought by catalogue renaming can be effectively solved, and the distributed file system metadata management method has the advantages of high storage performance, small maintenance expenditure, high load, no bottleneck, good expansibility and balanced load.

The invention discloses a method for distributing metadata of a distributed type file system. The method uses a catalogue of the file system as a basic unit of Hash, adopts an extensible Hash method to distribute the metadata to each metadata server to ensure that clients can position the position of the metadata; the metadata has high accessing efficiency, even distribution and balanced load; meanwhile, the method maintains storage locality of the catalogue, realizes conveniently prefetching, and improves the accessing efficiency. A unique and unchanged identifier is given to the catalogue to avoid metadata migration caused by the path name of Hash and improve the system performance; participation or exit of the metadata server cannot cause redistribution of mass metadata so as to furthest reduce the metadata migration and ensure high expandability.

To provide an apparatus and method for efficiently displaying image content, which is sent and received between a server and a client, in a correct direction. An image content providing server stores, as property information, original angle information indicating correct directions of image content as metadata of the image content. The server generates content information including the original angle information and sends the generated content information to a client. This allows the client to understand the correct direction of the image content received from the server, and to obtain and display image data in the correct direction by performing rotation processing in the client device based on the obtained information or to request the server to perform the rotation processing.

A method, system and computer accessible medium for expiring access tokens in preparation for freezing file images. A metadata server may maintain a next scheduled quiesce time and may issue access tokens configured to expire before the next scheduled quiesce time. A metadata server may set an access token's expiration time to a maximum expiration time indicated by the next scheduled quiesce time or may set an access token's expiration time to a default expiration time if the default expiration time is earlier than the maximum expiration time. A storage device may recognize and enforce the access token's expiration time.

Managing file objects in a data storage system includes providing a plurality of metadata servers, each metadata server having information for only a subset of file objects in the storage system, each subset for each of the metadata servers being less than a total number of file objects for the system and includes providing at least one metadata location server, where an entity accessing a specific file object of the system determines which of the metadata servers contains data for the specific file object by first accessing the at least one metadata location server. Information for the subset of file objects may include a table having file identifiers and metadata location information. The metadata location information may point to data storage for a metadata object for a file.

A hierarchical storage data management system includes application servers, a metadata server interconnected with the application servers through a local area network, storage devices interconnected through data flow paths, and a storage network connecting the storage devices to the application servers and to the metadata server. The metadata server including a metadata management element, a data value management unit, and a hierarchical storage management element calculates a data value for each of stores data objects in the system, assigns a storage cost value for each of storage areas in the system, normalizes calculated data values and assigned storage costs to an identical value range, compares normalized data values with normalized storage costs thereby determining whether to relocate the data objects to different storage areas, and relocates data objects to storage areas with storage cost values identical with data values of the data objects.

The invention relates to a method and a system for dynamically managing metadata of a distributed file system in order to solve the problems that a metadata management method cannot dynamically adjust the distribution of the metadata according to the load change, smoothly extend a metadata server, process the access to a hot content and well deal with the access to a single hot file. The method comprises that: the metadata server dynamically divides the distribution of the metadata on each metadata server according to the access condition of the metadata and manages the dynamically divided metadata according to the operation request of a client. The method and the system solve the problems of extensibility and load balance of the metadata server through dynamic division and the problem of consistency of the metadata through the management of the dynamically divided metadata.

In an aggregated file system, a file may begin with a set of stripe fragments all in the RAID-5 scheme in order to take advantage of the RAID-5 scheme's storage efficiency. After that, when one of the fragments is accessed by a file switch, it will be duplicated into the data mirroring scheme. The file's corresponding metadata server maintains a data structure, e.g., a bitmap, indicating which fragments have been duplicated into the data mirroring scheme. In other words, the file, at this moment, exists in a hybrid scheme. A file consolidator running on the metadata server is triggered at a predefined time to copy the fragments from the data mirroring scheme back to the RAID-5 scheme, This file consolidator also updates the bitmap to reflect the changes to the file's scheme change. This hybrid scheme is expected to increase the I / O capacity of the conventional RAID-5 scheme and the storage usage of the conventional mirroring scheme.

The invention discloses a system for realizing efficient safe storage and a method thereof. The invention comprises the steps as follows: a storage server, a protocol server, a coding server, a decoding server and a metadata server are arranged; during writing operation, the protocol server writes initial data for writing into the storage server and at the same time indicates the coding server to encode the data for writing, the coding server encodes the data to generate redundant data and assigns the redundant data to the appointed storage server, and after writing is finished, basic information of the written file data is stored in the metadata server; during reading operation, the protocol server accesses the metadata server to obtain the storage position of the initial data for reading and obtains the data from the storage position of the data to be combined into the file data; when the initial data can not be read normally, the metadata server is accessed, the storage position of the redundant data is obtained, and the redundant data is read into the decoding server. The invention can effectively improve the efficiency and the reliability of data storage.

Characters of the method are that client proxy software of parallelized file system on each front-end host implements supporting VFS access; server of metadata is in charge of executing increment scaling, transporting management and rate control i.e. operations so as to realize performance guarantee of front-end application in hierarchical storage system for mass data (HSSMD). According to performance, data servers are divided into fast data servers and slow data servers. Data server processes incremental scan command sent from metadata server, returns scanned information, and meanwhile executes file transfer command sent from metadata server. Based on condition of load, the method dispatches procedure of data transfer. Based on condition of remnant space (RS) of fast device, the method carries out degradation initiatively. The invention reduces influence on application in front end from transporting procedure, raises rate of RS of fast device, and enhances self management for HSSMD.

Disclosed is a block chain packing and storing method.When multiple expenditures occur at a public key address, whether the balance is sufficient or not must be verified sequentially; expenditure addresses are classified according to address range, transactions can be verified respectively according to different threads or processes, and a same expenditure address can be ensured not to exceed the balance; the balance added by received money in each address range is transmitted to a server that the corresponding address range is in, and final balance is calculated again; during data storing, in order to overcome limitation of disk writing speed, disk writing can be performed on multiple servers circularly and sequentially, division modulus-taking mapping can be performed on block chain block height and number of the servers, or a corresponding relationship between the block height and the servers can be used as metadata to be managed by a special metadata server; during data accessing, the metadata server is accessed first, the server corresponding to the block height is acquired, data size of disk writing in each time is set, and time when disk writing of each time is finished can be made not to be greater than time when disk writing of next time should be started.

The invention provides a distributed key-value query engine system and a query method. The system comprises a metadata server, at least one proxy node and at least one storage node, wherein the metadata server is used for maintaining the information of a global routing table pointing at a storage node; the proxy node comprises a query forwarding module and a data forwarding module, wherein the query forwarding module is used for forwarding a client request to a corresponding storage node in response to a request including a target KEY sent from a client based on the information of the global routing table, and the data forwarding module is used for transmitting a response packet including a target Value, returned from the corresponding storage node, to the client; and the storage node is used for acquiring the needed data from a local storage based on the received client request and transmitting the response packet including the target Value to the proxy node. According to the distributed key-value query engine system and the query method, the key-value data can be stored and queried in a distributed manner, massive key-value data can be accommodated and the adaptability to highly concurrent access is better.

The invention discloses a rapid data memory method for improving concurrent visiting performance in a cloud memory system. The cloud memory system comprises a metadata server, data memory servers and clients, wherein only control signals are transmitted between the metadata server and the clients, and memory data flow is not transmitted between the metadata server and the clients; the memory data flow is transmitted between the clients and the memory servers; a data file which is bigger than a threshold value is uniformly divided into various data blocks which are stored on each data memory server in a distributed way; and single data file can be concurrently read and written.

This invention relates to a graded storing method for mass data characterizing that the customer proxy software of parallel file systems on front hosts realizes support to VFS access by system interface sub-modules and sub-modules of the VFS layer, a metadata server is responsible for organizing data files on different data servers to a unified parallel file system view, and a metadata management module provides operations of accessing metadata, a file migration deciding module gets file access information from the data server periodically and makes decision to file migration according to file system load and grade situation of equipment and an executing module of the data server executes migration.

A distributed object-based storage system and method includes a plurality of object storage devices for storing object components, a metadata server coupled to each of the object storage devices, and one or more clients that access distributed, object-based files on the object storage devices. A file object having multiple components on different object storage devices is accessed by issuing a file access request from a client to an object storage device for a file object. In response to the file access request, a map is located that includes a list of object storage devices where components of the requested file object reside. The map is stored as at least one component object attribute on an object storage device. The map is sent to the client which retrieves the components of the requested file object by issuing access requests to each of the object storage devices on the list.

The invention discloses a data access method, a data access device and a data access system. The method comprises the following steps that: an access request containing a block number identifier is received; a pre-established local block unit searches the corresponding relation between the block number of the local block unit and storage node resource information providing storage services according to the block number to obtain a storage node position corresponding to the block number identifier and a block data identifier; and the local block unit sends the data access request to the storage node according to the storage node position corresponding to the block number identifier and the block data identifier, wherein the storage node resource information comprises the storage node position information and the block number identifier of a storage node. By the method, the device and the system, the problem that a metadata server becomes a bottleneck of the system because the current distributed file system is required to address through the metadata server at every access is solved, and the access efficiency is low is overcome.

The invention relates to a method and device for accessing files and a distributed storage system. The method includes: receiving a file uploaded by a client; sending a file index obtaining request to a metadata server cluster and obtaining a metadata index returned by the metadata server cluster; sending the file or the slicing of the file to a selected data server cluster for storage; and sending the storage index of the file or the slicing of the file in the selected data server cluster, the identifier of the selected data server cluster and the metadata index to the metadata server cluster for associate storage. According to the method and device for accessing the files and the distributed storage system, since the data is stored in the data server clusters of different data centers, the operating cost of the distributed storage system of the data center can be reduced.

A distributed storage system has MDSs (metadata servers). Directories of file system namespace are distributed to the MDSs based on hash value of inode number of each directory. Each directory is managed by a master MDS. When a directory grows with a file creation rate greater than a preset split threshold, the master MDS constructs a consistent hashing overlay with one or more slave MDSs and splits directory entries of the directory to the consistent hashing overlay based on hash values of file names under the directory. The number of MDSs in the consistent hashing overlay is calculated based on the file creation rate. When the directory continues growing with a file creation rate that is greater than the preset split threshold, the master MDS adds a slave MDS into the consistent hashing overlay and splits directory entries to the consistent hashing overlay based on hash values of file names.

A crash recovery system and method for distributed file server using object based storage are provided. The system includes: a client for accessing a file system using an object-based storage device (OSDFS), transmitting a command to an object-based storage device (OSD) and accessing a metadata server (MDS); a network for providing an interface and transferring data between the client, the metadata server and the object-based storage device; an object-based storage device for analyzing the command from the client and performing corresponding operations of the command; and a metadata server for storing and managing metadata controlling a direct access to a predetermined file from the client to the object based storage device in order to provide the metadata to the client, and checking and recovering a consistency of the stored and managed metadata when the OSDFS is malfunctioned.

An apparatus, system, and method are disclosed for managing storage space allocation. The apparatus includes a recognizing module, a reserving module, and a managing module. The recognizing module recognizes a trigger event at a client of the data storage system. The reserving module reserves logical units of space for data storage. The management module manages the logical units of space at the client. Such an arrangement provides for distributed management of storage space allocation within a storage area network (SAN). Facilitating client management of the logical units of space in this manner may reduce the number of required metadata transactions between the client and a metadata server and may increase performance of the SAN file system. Reducing metadata transactions effectively lowers network overhead, while increasing data throughput.

A data processing system includes compute nodes, at least one metadata server, iSCSI storage access servers, and data storage devices. The metadata server services requests from the compute nodes for file mapping information identifying iSCSI LUNs and logical blocks of file data. The storage access servers service iSCSI I / O requests from the compute nodes, and report server loading to the metadata server. A Gigabit Ethernet IP network transfers read and write data between the compute nodes and the storage access servers. The storage access servers are linked to the data storage devices for parallel access to iSCSI LUNs of the file data in the data storage devices. The metadata server is programmed for server load balancing by indicating to the compute nodes respective ones of the storage access servers that should be used for access to the iSCSI LUNs.

A cluster of computer system nodes share direct read / write access to storage devices via a storage area network using a cluster filesystem and operating system implementing DMAPI. Threads executing on a metadata client know when a DMAPI event is required, and generate the DMAPI event on their own initiative when necessary. A metadata server maintains DMAPI queues. If the metadata server relocates to another host, the DMAPI events in the DMAPI queues are moved transparently to users.