99 results about "Defragmentation" patented technology

A defragmentation method may include: acquiring at least one chunk information of data in response to a writing request for the data, wherein the at least one chunk information may include an identification of at least one chunk and a stream identification assigned to data of the at least one chunk; updating a fragmentation degree of each of the at least one chunk and a stream to which the each of the at least one chunk belongs in a chunk index structure to obtain an updated chunk index structure, based on the at least one chunk information, wherein the chunk index structure may include identifications of a plurality of chunks, stream identifications corresponding to the plurality of chunks, and fragmentation degrees of the plurality of chunks; and determining a first chunk to be defragmented, based on the fragmentation degrees of the plurality of chunks in the updated chunk index structure.

The invention discloses a repeated data deletion method and system based on ZNS SSD, and belongs to the field of repeated data deletion.The method comprises the steps that a fingerprint index table and a fingerprint updating cache are maintained, three-time opportunity repeated data deletion is achieved, and repeated data search is conducted in the fingerprint index table, the fingerprint updating cache and an address mapping table respectively; an address mapping table, an address reverse mapping table and a reference counting table are maintained, physical space garbage collection / defragmentation and write request merging are achieved, the address mapping table, the address reverse mapping table and the reference counting table conduct garbage collection with partition as granularity when free space is insufficient, and a fingerprint index table is not updated after garbage collection data migration; and instead, the mapping of fingerprints to migration-in physical pages is stored in a fingerprint updating cache, the physical pages are sequentially distributed in partitions by the fingerprint updating cache, normal data write requests and garbage collection data migration write requests are respectively cached, and the write requests which are continuous in sequence are merged and then issued. According to the method, the high-performance and low-cost efficient storage system can be constructed on the basis of the ZNS SSD at relatively low overhead.

The invention provides a data defragmentation method and a storage device. The method comprises the steps that a plurality of candidate logic intervals are determined from a plurality of logic intervals according to access popularity information, and the access popularity information reflects host access popularity corresponding to the logic intervals respectively; after the plurality of candidate logic intervals are determined, a first logic interval in the plurality of candidate logic intervals is determined as a target logic interval according to fragment level information, and the fragment level information reflects data fragmentation degrees respectively corresponding to the plurality of candidate logic intervals; and performing defragmentation on the target data belonging to the target logic interval. Therefore, the operation efficiency of the storage device can be improved.

Techniques are described for reducing capacity fragmentation by using an Automatic Defragmentation Service (ADS). More particularly, hypervisors (HVs) that are candidates to defragment are identified, an HV to defragment is selected, and one or more VM instances are migrated from the selected HV to a different HV. According to certain implementations, instead of migrating a VM to a new HV, the VM is live migrated to an existing HV.

The invention provides a data defragmentation method and a data storage system. The method comprises the steps that a plurality of candidate logic intervals are determined from a plurality of logic intervals according to access popularity information, and the access popularity information reflects host access popularity corresponding to the logic intervals respectively; after the plurality of candidate logic intervals are determined, a first logic interval in the plurality of candidate logic intervals is determined as a target logic interval according to response time information, and the response time information reflects storage device response time corresponding to the plurality of candidate logic intervals respectively; and instructing the storage device to perform defragmentation on the target data belonging to the target logic interval. Therefore, the operation efficiency of the storage device can be improved.

The invention discloses a bandwidth fragment self-adaptive arrangement method and device based on a DT algorithm, and belongs to the technical field of power system communication, and the method comprises the steps: obtaining basic data of a target optimization network, and building a digital twinborn body of the target optimization network; generating a data set based on the digital twin by referring to the logic of agent and environment interaction in reinforcement learning; preprocessing the data set to obtain a training data set; building a fragment adaptive arrangement model of the target optimization network based on a DT algorithm; carrying out model training on the fragment self-adaptive arrangement model by utilizing the training data set; and carrying out bandwidth defragmentation on the target optimization network by using the trained defragmentation adaptive defragmentation model. According to the method, the state of the target network can be quickly and accurately reflected, and the problem of fragments in the network can be adaptively processed, so that the utilization rate of network resources is effectively improved, and the degree of network congestion is reduced.

The invention discloses a defragmentation method and device, electronic equipment, a storage medium and a program product. According to the implementation scheme, the method comprises the steps of determining a defragmentation transaction, and a to-be-defragmented table and a to-be-defragmented B + tree corresponding to the defragmentation transaction; blocking the to-be-sorted table, and creating a temporary table corresponding to the to-be-sorted table; executing degradation processing on blocking of the to-be-sorted table, and inserting the to-be-sorted data into the temporary table to generate an inserted temporary table; operation corresponding to the logic log is redone on the inserted temporary table, and a redone temporary table and a redone temporary B + tree are obtained; and exchanging the to-be-sorted B + tree corresponding to the to-be-sorted table into the remade temporary B + tree to obtain a fragment sorting table. When the defragmentation transaction is executed, exclusive blocking does not need to be executed on the to-be-sorted table in the whole process, the to-be-sorted table is made to be in the available state, defragmentation of the to-be-sorted table is completed, and the concurrency and availability of a database system are improved.

The invention discloses an embedded file system implementation method based on SPI Flash, which comprises the following steps: reading a super block during initialization, and establishing a state mapping table of file record items and data sectors; when the file is opened, quickly matching record items through file name metadata, and if the matching fails, allocating new items; during reading and writing, aggregating data through the write cache, asynchronously updating the data to the flash memory and refreshing the mapping table; if a change occurs when the file is closed, distributing a new space and marking the original space as failure to avoid real-time erasing; and finally, garbage collection and defragmentation are executed in the idle period of the system. According to the method, the direct access to the SPI Flash is reduced through the memory mapping table, the erasing frequency is reduced by utilizing the write cache and an asynchronous mechanism, and the time-consuming erasing operation is transferred to the background, so that the processing efficiency of file operation and the real-time performance of the system are remarkably improved, and meanwhile, the service life of a storage medium is prolonged.

Methods, systems, and devices for data defragmentation control are described. A memory system may include one or more regions of logical addresses and a plurality of memory cells arranged according to a plurality of physical addresses. In some instances, data may be stored to one or more discontinuous physical addresses and it may be desirable rearrange the data to be within continuous physical addresses (e.g., it may be desirable to defragment the data). Accordingly, the data stored to the one or more discontinuous physical addresses may be arranged (e.g., rearranged) to be within continuous physical addresses based at least in part on a value stored to one or more registers of the memory system.

A defragmentation method may include: acquiring at least one chunk information of data in response to a writing request for the data, wherein the at least one chunk information may include an identification of at least one chunk and a stream identification assigned to data of the at least one chunk; updating a fragmentation degree of each of the at least one chunk and a stream to which the each of the at least one chunk belongs in a chunk index structure to obtain an updated chunk index structure, based on the at least one chunk information, wherein the chunk index structure may include identifications of a plurality of chunks, stream identifications corresponding to the plurality of chunks, and fragmentation degrees of the plurality of chunks; and determining a first chunk to be defragmented, based on the fragmentation degrees of the plurality of chunks in the updated chunk index structure.

A host Flash Translation Layer (FTL) synchronizes host FTL operations with the drive FTL operations to reduce write amplification and over-provisioning. Embodiments of FTL synchronization map, at the host FTL software (SW) stack level, logical bands in which data is managed, referred to as host bands, to the physical bands on a drive where data is stored. The host FTL tracks validity levels of data managed in host bands to determine validity levels of data stored in corresponding physical bands, and optimizes defragmentation operations (such as garbage collection processes and trim operations) applied by the host FTL SW stack to the physical bands based on the tracked validity levels.

The invention relates to the technical field of data processing, in particular to a multi-dimensional interactive control system based on digital human demonstration, and the system comprises a state collection module which is used for calculating the space mapping relation between skeleton node displacement and joint angle values, connecting the displacement data of each skeleton node with the corresponding joint angle value, and generating a multi-dimensional state vector; and carrying out operation on displacement data of each skeleton node in the multi-dimensional state vector to generate an action feature mapping graph. According to the method, by calculating the space mapping relation between the bone node displacement and the joint angle value, the multi-dimensional state vector is generated, the action feature mapping graph is constructed, and collection and expression of human body action data can be achieved; and block division and address allocation of the memory space are performed based on the displacement data of each skeleton node in the action feature mapping graph, and memory defragmentation is executed to generate a dynamic resource allocation table, so that the system operation efficiency and the resource utilization rate are improved.

Methods, systems, and devices for increased file loads using file based optimization (FBO) and prefetch within a memory system are described. A system may support using a prefetch command to prioritize prefetching for artificial intelligence (AI) related data (or other high priority data). A host system may transmit a prefetch command for the AI-related data before performing one or more AI operations that use the data. In response, the memory system may prefetch the AI-related data and store the data in a cache. The storage of the AI-related data in the cache may be prioritized over other data types associated with lower priorities. The host system may transmit an FBO command to defragment the AI-related data in the memory system prior to the prefetch, which may reduce latency of the prefetch.

A method for defragmentation management, the method includes (a) moving, by a defragmentation process related to a set of database entries, content of a current subset of the set to a next subset of the set; (b) marking the current subset as being moved and linking the current subset to a location of the next subset; (c) receiving a user transaction write request for writing user data related to the current subset, following the moving; (d) writing the user data to the current subset and to the next subset, before the current subset is deleted; and (e) deleting the current subset when a start time of an oldest transaction follows a time of commit of the defragmentation process.

This application provides a memory defragmentation method performed by a computer device. The method includes: obtaining n idle blocks before a tail idle block in a memory of a graphic processing unit, the tail idle block located at a head of remaining unallocated idle blocks in the memory, and n being a positive integer; obtaining tail non-idle blocks in the memory, the tail non-idle blocks comprising one allocated block or a plurality of continuous allocated blocks immediately before the tail idle block in the memory; determining one or more target idle blocks in the n idle blocks, a size of the target idle block being not less than a size of the allocated blocks among the tail non-idle blocks; migrating data in the tail non-idle blocks to the target idle blocks; and updating the head of remaining unallocated idle blocks in the memory by releasing the tail non-idle blocks.

The application provides a resource fragment consolidation method, a controller, a control node and a computer cluster, and relates to the technical field of data processing. The method simulates resource scheduling according to a first resource capacity and a first task currently being executed to obtain a plurality of candidate resource scheduling paths; determines a target resource scheduling path from the plurality of candidate resource scheduling paths; and schedules the first task based on the target resource scheduling path to perform resource fragment consolidation. After each candidate resource scheduling path is used to perform resource fragment consolidation, the residual resource capacity of at least one node can reach the first resource capacity. That is, the method can actively eliminate resource fragments in the computer cluster by scheduling the first task to perform resource fragment consolidation. Compared with the method of slowing down the generation of new resource fragments from the source, the scheduling reliability of a task with high resource demand can be effectively improved.

The application relates to the technical field of big data analysis, and discloses an OT / IT heterogeneous data unified access and standardized conversion method, which comprises the following steps: acquiring a data stream and extracting service load data and metadata therefrom; converting the metadata into fixed-length hash features by using hash operation, stripping the metadata, and encapsulating the service load data and the hash features; dividing a plurality of continuous memory buffer areas corresponding to a bit mask base number of a memory space, and extracting feature offsets of the hash features by using the bit mask; sequentially appending and writing data into the buffer areas according to receiving time; and sequentially writing the data into a storage medium in batches when the memory occupation reaches a preset threshold. The application converts the device logical owner relationship into the continuity of the medium physical block by using the hash routing, suppresses write amplification, avoids system lagging caused by fragment consolidation, and guarantees the continuous and smooth data collection of a data surge.

Aspects of the disclosure provide methods, apparatuses, and non-transitory computer-readable storage mediums for receiving media data. One apparatus includes processing circuitry that receives a timed metadata track of a plurality of segments including a plurality of metadata samples. A duration of each of the plurality of segments is equal to a duration of a metadata sample included in the respective segment. A correct timing of each of the plurality of metadata samples is maintained during fragmentation and defragmentation processes that are performed on the timed metadata track. The processing circuitry determines a presentation time of one of the plurality of metadata samples. The processing circuitry processes event information included in one or more event message boxes included in the one of the plurality of metadata samples based on the presentation time of the one of the plurality of metadata samples.

The invention discloses a data writing method, device and equipment and a computer readable storage medium, which are applied to the technical field of data storage, and the method comprises the following steps: loading a secondary queue list; the second-level queue list comprises a first-level candidate queue and a second-level high-frequency acknowledgement queue; selecting a first target block and a second target block from the blocks; the updating frequency of the data in the first target block is higher than the updating frequency of the data in the second target block; determining whether a current logical block address is in a secondary queue table when it is determined that the data is written in the block to the logical block address; and if the current logic block address is in the secondary queue list, writing the current logic block address into the first target block, otherwise, writing the current logic block address into the second target block. Compared with the mode of randomly storing the data into the data blocks at present, the method has the advantages that the data with low updating frequency and the data with high updating frequency are respectively written into the different target blocks as far as possible on the basis of the secondary queue list, so that the invalid data in the blocks with low updating frequency are received very little, and the frequency of garbage collection and fragment arrangement operation is remarkably reduced.

The present application relates to the field of cloud computing container platform, and discloses a container cluster resource fragmentation arrangement method, device, equipment and storage medium, the method comprises collecting node multidimensional resource utilization index data, and constructing a dynamic coloring model;The multidimensional resource utilization index data is input into the dynamic coloring model to obtain the node fragmentation index calculated by the model;Get the node fragmentation index threshold, compare the node fragmentation index with the node fragmentation index threshold, screen out the initial node to be arranged, calculate the arrangement investment return rate of each initial node to be arranged, select the node with a return rate greater than 1 as the target node to be arranged;Determine the arrangement window period;In the arrangement window period, the Pod migration problem of the target node to be arranged is modeled as a multidimensional packing problem, and the to-be-migrated Pod is obtained through an optimization algorithm, and the target node to be arranged is arranged based on the minimum migration cost principle.The present application solves the problem of resource fragmentation recovery lag and low cluster utilization in the prior art.

A data storage device having a flash translation layer configured to handle file-system defragmentation in a manner that substantially avoids physical data movement in a flash memory. In an example embodiment, a memory controller operates to update a logical-to-physical table thereof to change association of physical addresses of sections of user data from being associated with source logical addresses to being associated with destination logical addresses of the host defragmentation requests without moving the user data in the flash memory. Such updates can reduce the number of instances in which the host addresses a non-contiguous logical-address range, which results in a beneficial reduction of the number of input / output commands sent to the data storage device and of the associated processing overhead.

One example method includes identifying a cloud object as a potential candidate for defragmentation, evaluating the cloud object to determine what portion of segments of the cloud object are expired, when the portion of expired segments meets or exceeds a threshold, segregating the expired segments and unexpired segments of the cloud object, creating a first new cloud object that includes only unexpired segments, creating a second new cloud object that includes only expired segments, and deleting the cloud object from storage.

The present disclosure relates to a device and method for storing duplicate data blocks. Specifically, the present disclosure provides a device for eliminating the need for defragmentation in a deduplication backup system. The device is configured to store one or more compressed containers, wherein each container includes a plurality of fragments, wherein one or more duplicate data blocks are stored in the plurality of fragments of the one or more containers. The device is also configured to delete the first duplicate data block stored in the fragment by decompressing the fragment, replacing the first duplicate data block with zeros in the fragment, and recompressing the fragment. In this way, a solution is provided to delete obsolete data without metadata updates.

The application provides a memory fragment arrangement method and device, electronic equipment and storage medium, and relates to the technical field of storage. The memory fragmentation index is determined by the proportion of the free pages in each partition that do not meet the memory allocation request in the total free pages in the partition and the memory weight factor of each partition, so that the memory fragmentation index is more consistent with the actual use of the memory, so that the memory fragment arrangement is performed when the memory fragmentation index is greater than the preset arrangement threshold, which is more consistent with the arrangement expectation of the memory, so that the memory arrangement is actively performed in time, the case of excessive memory fragments in the memory is avoided, the probability of meeting the memory demand of the application program is improved, and the normal operation of the application program is ensured.

A defragmentation process is provided for contiguous persistent or volatile memory regions. The defragmentation process selects and moves extents, updates extent maps, and ensures all read / write operations are consistent and uninterrupted. The defragmentation process can be applied to online maintenance defragmentation, online on-demand defragmentation, or offline defragmentation. The illustrative embodiments provide a source-destination mapping algorithm that allows for optimal defragmentation outcome with least amount of space relocation. In some embodiments, a cost-based greedy algorithm is used for source-destination mapping. Quiesce and unquiesce mechanisms allow for fine-grained access control for the extent currently being relocated by defragmentation.

This application discloses a defragmentation method, device, and medium for GoldenDB databases, relating to the field of database data processing technology. The method includes: obtaining metadata information of all tables within the cluster from the management node of the GoldenDB distributed database; filtering tables that meet the defragmentation conditions from all tables based on preset dual thresholds to accurately determine the defragmentation targets; intelligently grouping the tables to be defragmented according to a preset defragmentation strategy and generating corresponding defragmentation tasks based on the grouping results; and distributing the defragmentation tasks to the computing nodes of the GoldenDB database for execution to complete the data defragmentation operation within the cluster. Through parallel execution by grouping, the total defragmentation time is reduced compared to existing random parallel schemes. The dual threshold filtering based on table size and fragmentation rate ensures that only tables requiring defragmentation are processed, saving system resources.

In certain aspects, the memory controller includes a controller memory for storing a logical-to-physical (L2P) address mapping table corresponding to a file, and a controller processor configured to control a memory device, receive a mapping update command, and update the L2P address mapping table according to the mapping update command by replacing original logical addresses of logical block address (LBA) segments of the file with new continuous logical addresses of a merged LBA segment of the file, and changing an original mapping relation between the original logical addresses of the LBA segment of the file and physical addresses of the file, to a new mapping relation between the new continuous logical addresses of the merged LBA segment of the file and the physical addresses of the file.

A method for transaction conflict management, the method includes writing, by a transaction controller and to a given logical address within a logical address space supported by a storage system, defragmentation data related to a defragmentation process, the defragmentation data being overwritable; identifying, by transaction controller, a transaction conflict associated with the defragmentation process, and (b) one or more user transaction requests related to the given logical address; and resolving the transaction conflict based on (i) the defragmentation data being overwritable, (ii) a timing of the writing of the defragmentation data, and (iii) one or more timings of one or more user transaction operations in relation to the one or more user transaction requests.

The invention provides an object defragmentation method, device and equipment and a storage medium, and is applied to the technical field of object storage, and the method comprises the steps: obtaining the number of update times of data in a preset address space in a cache pool in a previous unit time, determining the space replacement frequency of the cache pool in the previous unit time according to the update times in the previous unit time; obtaining a first data volume written through to the data pool by the front-end server in the previous unit time and a total second data volume written in, and determining a write-through proportion of the data pool according to the first data volume and the second data volume; determining a target value currently corresponding to each parameter of the object defragmentation task according to the space replacement frequency and the write-through proportion; the target value of each parameter is matched with the current writing service pressure condition of the front-end server; and executing an object defragmentation task according to the target value currently corresponding to each parameter. By adopting the technical scheme of the invention, the space requirement and / or the performance requirement required by the sudden task of the front-end server can be met.