906 results about "Shard" patented technology

A method of maintaining a solid-state drive so that free space within memory blocks of the drive becomes free usable space to the drive. The drive comprises cells organized in pages that are organized in memory blocks in which at least user files are stored. A defragmentation utility is executed to cause at least some of the memory blocks that are partially filled with data and contain file fragments to be combined or aligned and to cause at least some of the memory blocks that contain only invalid data to be combined or aligned. A block consolidation utility is then executed to eliminate at least some of the partially-filled blocks by consolidating the file fragments into a fewer number of the memory blocks. The consolidation utility also increases the number of memory blocks that contain only invalid memory. All of the memory blocks containing only invalid data are then erased.

When a dynamic data structure is used for managing sparsely accessed data stored in memory of a digital computer, pages of the memory are dynamically allocated and de-allocated to respective portions of the dynamic data structure so that the pages become distributed over the portions of the dynamic data structure and the de-allocated pages include free pages that are mapped in the dynamic data structure and free pages that are not mapped in the dynamic data structure. To reduce memory fragmentation and recover memory, upon de-allocating a page of memory from a portion of the data structure, a determination is made whether or not to un-map the de-allocated page from the portion of the dynamic data structure so that un-mapping of the de-allocated page has a probability that is the inverse of a superlinear function of the number of allocated pages in the portion of the dynamic data structure.

Provided are apparatus and method for distributedly storing an object in a plurality of data centers included in a cloud storage system. In the method, an object may be divided into a plurality of sub-objects each having a fragmentation size. Then, the sub-objects may be distributedly stored in at least one data center based on weights of the plurality of data centers.

Techniques for fragmenting a file or a collection of media data are disclosed. According one aspect of the techniques, a file pertaining to a title is fragmented into a header and several tails or segments. The header is a continuous portion of the file while the segments are respective parts of the remaining portion of the file. The header is seeded substantially in all boxes, and none, one or more of the segments are distributed in each of the boxes in service. When a title is ordered, the header is instantly played back while the segments, if not locally available, are continuously fetched respectively from other boxes that have the segments.

The invention relates to a big data technology-based space data storage processing middleware framework. The framework can provide a method for rapidly obtaining data content from mixed data of existing multi-source heterogeneous structural data and non-structural data; and distributed storage efficiency is improved by adopting a mainstream big data access tool. The big data technology-based space data storage processing middleware framework disclosed by the invention performs extraction, conversion and loading on the multi-source heterogeneous space data to establish a diversified, fragmented and non-structural data distributed virtualized storage framework, so that directly readable data content is provided for subsequent space big data analysis and mining.

A defragmentation method and system to overcome fragmentation of virtual volumes in storage area networks (SANs). The method includes combining and migrating fragments of data spread over multiple storage areas in order to increase storage space utilization and optimize access time to storage. By defragmenting virtual volumes both performance and storage utilization is significantly improved.

The invention provides a system combining intelligent knowledge diagnosing and teacher online tutoring. Basic information of a student is acquired first, required information is extracted from a massive resource library according to the information of the student so as to perform intelligent test paper composition, an intelligent diagnosing report is generated after the answers of the student are acquired, and learning resources corresponding to the weak knowledge links of the student are pushed to the student; the student can also transmits the intelligent diagnosing report to an online teacher for one-to-one communication, the teacher analyzes the diagnosing result again to increase the accuracy of the result, and the teacher can also push targeted resources to the student after the diagnosing. The method has the advantages that the convenience of the Internet is fully utilized, the student's time is utilized in a fragment manner, comprehensive intelligent knowledge diagnosing can be provided for the student fast, the student can find the weak knowledge links conveniently, and learning efficiency of the student is increased.

The embodiment of the invention discloses a page editing method, a page editing device and a page editing system. The page editing method comprises the following steps of: opening a parent window in a page editing system, and sending an editing request to an editing server through the parent window; after the parent window receives editing information which is returned by the editing server, acquiring a page to be edited from a service server through a child window which is opened in the parent window according to the editing information, and loading the page to be edited in the child window; editing the block information of the page to be edited and page fragments in each block in the child window, and thus obtaining an edited page; and after the page is edited, sending the edited page to the editing server through the parent window, so the editing server combines the edited page which is received to obtain a page file and synchronizes the page file with the service server. By the invention, WYSIWYG page editing and previewing can be realized, so the threshold of a portal page is reduced, and process complexity is reduced.

The invention belongs to the technical field of network storage, and discloses a method for creating an allocation-on-demand incremental volume. The volume has a two-layer volume structure, and realizes separation of a physical space and a logical space. The creating method comprises that: a user state interface participates in a new parameter (N) in a user state command lvcreate of a logical volume manger (LVM). The real volume of the bottom layer is a common logical volume, and the virtual volume of the upper layer is a data structure in a Linux kernel. The physical space of the real volume is allocated according to the actual requirement, so the utilization rate of a disk space is improved. Real-time monitoring of the physical space of the real volume and automatic capacity expansion of the actual space effectively avoid overflow of the spaces. A user can randomly designate capacity expansion increment magnitude as required to perform capacity expansion of the virtual volume. The volume processes the write request by adopting a mode of additional write. Moreover, the volume processes garbage fragments by adopting a quick and effective garbage reclamation algorithm.

The invention relates to the field of a distributed system, in particular to a resource scheduling technology in the distributed system. According to a task allocation method, a share of allocated resources of a user is obtained; a to-be-allocated task is selected from a to-be-allocated task list; based on a highest threshold, the to-be-allocated task is allocated to a first computing node with residual resources capable of meeting the to-be-allocated task; and after the to-be-allocated task is allocated to the first computing node, at least one monitored resource exists in the first computingnode, and the residual quantity of the monitored resources is greater than or equal to the highest threshold corresponding to the monitored resources. Through the scheme provided by the task allocation method and apparatus, resource fragments in the distributed system can be reduced, so that the resource utilization rate of the distributed system is increased.

The invention relates to an optimized deploying method based on virtual cluster online migration, and belongs to the cloud computing technical field; the method comprises the following steps: using a management module to connect with a management cluster node, wherein the management module is provided with an overall monitoring module, an algorithm forming module, a migration planning module and a migration driving module; using the overall monitoring module to receive the cluster node and virtual machine information data in the cloud computing environment, and sending the information to the algorithm forming module so as to predict future loads and build a long term load performance model; planning a deploying strategy for the virtual cluster, and sorting the virtual machines in the virtual cluster into different host nodes of the cluster, thus allowing the virtual machines to fully utilize the host resources in the cluster when the virtual cluster migration is finished. The novel method can solve the problems that the virtualization technology may easily cause resource fragments, and can solve the defects that the virtual machines cannot obtain the expected requesting resource quantity when the dynamic loading demands and the virtual machines are combined, thus reducing virtual machine performance loss in the cluster nodes, and improving resource utilization rate.

File carving is a technique whereby data files are extracted from a digital device without the assistance of file tables or other disk meta-data, and can therefore be used to recover files that are fragmented. Sequential hypothesis testing procedures are used to detect a fragmentation point of a file by sequentially comparing adjacent pairs of blocks from the starting block of a file until the fragmentation point is reached. The detected fragmentation point can be used to help recover the fragmented file. Such a serial analysis helps to minimize errors and improve performance.

According to one embodiment, distributing data across a plurality of storage shards can comprise generating a file key for each file of a plurality of files stored in a plurality of physical shards, each physical shard maintained by a node of a plurality of nodes in one or more clusters. The file key can comprise a hash of an enterprise identifier for an entity to which the creator of the file is a member, a hash of a folder identifier for a location in which the file is stored, and a hash of a file identifier uniquely identifying the file. The generated file keys can be sorted into an ordered list and the ordered list can be logically partitioning into a plurality of logical shards. Each logical shard of the plurality of logical shards can then be mapped to one of the plurality of physical shards.

In a blockchain networking environment, a wallet keying system creates a private key that is used for authorizing user transactions while negating the need for the private key to be hosted on a user device or in a user application such as a blockchain wallet. The private key is fragmented and stored in a distributed fashion to prevent unauthorized access. To ensure scalability and network performance, a shard system allows the movement of users and user applications and tools within a decentralized and distributed blockchain network to enable better network balance and utilization of available resources at the local node level. A network provisioner is permitted to impose and adjust restrictions applied to a wallet or on blockchain tokens with the restrictions either being adjusted based on milestones and measured parameters or manually.

The invention provides a high-performance file storage and management system based on an HDFS. A file preprocessing module and a file merging module are provided for overcoming the defect that an HDFS(Hadoop Distributed File System) supports small file storage, a pre-fetching cache module is provided for improving the file reading efficiency, a fragmentation module is provided for improving the overall space utilization rate, and a data encryption module is provided for protecting private files of a user. The invention provides a universal small file merging strategy and an HDFS client optimization scheme, and combines the advantages of an HAR file archiving method and a MapFile method so as to be suitable for any type of files. Specifically, the system optimizes a small file merging strategy and provides a defragmentation mechanism which is easy to update so as to improve the space utilization rate of an HDFS cluster. Meanwhile, a hotspot file strategy based on frequency statistics improves the reading speed of data, and improves the safety of a user account and a private file based on MD5 login encryption and AES file encryption.

The invention discloses a secret key generation method, a signature method, a signature verification method, a signature verification device, equipment and a medium, which relates to the technical field of computer encryption technologies and block chains, and can be used in cloud computing and cloud fields. According to the specific implementation scheme, the method comprises the following steps:determining a first local secret of a local node, segmenting the first local secret into secret fragments according to the number of the multi-party nodes, and transmitting the secret fragments to each node in the multi-party nodes in a one-to-one correspondence manner; and obtaining secret fragments segmented and transmitted by other nodes in the multi-party node, and generating a private key and a public key of an asymmetric key pair according to the obtained secret fragments of the multi-party node. According to the invention, the security and privacy of key generation can be improved.

The invention discloses a blockchain storage method supporting incremental updating of Internet of Things data. The method comprises the following steps of partitioning data, calculating a MD5 value after the data is processed into fragmented data, and sending the processed MD5 value to a full node domain; calling the intelligent contract by all nodes, and executing an adaptive PoW algorithm to complete an uplink operation; transmitting the fragmented data to a storage domain for storage, and sending a storage certificate to a full node domain after the storage node successfully stores the fragmented data. In the data updating stage, the edge nodes compare MD5 values of the fragmented data before and after modification, the fragments with data change are screened out and uploaded, and waste of storage space is reduced. According to the invention, incremental updating of data is realized, and consumption of storage space in the data server is effectively reduced. In addition, the difficulty-adaptive PoW algorithm is utilized to improve the transaction efficiency of the blockchain network.