30results about How to "Reduce write volume" patented technology

The invention discloses a data storage method. Data records are taken as basic read-write units for the data storage; and each data record occupies storage space of a virtual small sector and comprises keywords, data and a virtual sector distribution table representing the parameter characteristics of the data records. The data records are written into the storage space, the data of the data record to be written is compared with the data of the written data record of which the keyword is the same and the data is the latest base data; if differential writing is needed, the difference data is assembled, which is served as the data of the data record to be written according to the latest base data; otherwise, the data record to be written is directly written. In the data storage method, the differential data storage is considered in the storage process, the data writing amount is reduced, the occupation of the storage space is reduced and the writing speed is improved.

The invention discloses a distributed flash memory transaction processing method. The method comprises the following steps: S1, carrying out two-phase commit on a transaction, wherein state log records of the two-phase commit protocol are removed and internalized as flash memory metadata operations in interfaces of the transaction, and the end of the distributed transaction is marked with the final commit of a coordinator; S2, during the process of the transaction, storing a data temporary object as a shadow version, wherein a shadow mapping table is used for storing the address of the shadow version or the state of a page in the transaction processing, metadata of the page is used for recording information of the transaction, a transaction metadata page is used for recording the state of the transaction and a transaction state table is used for recording the address of the transaction metadata page; and S3, when the coordinator or a participant has a fault, scanning a part of the memory, reading the flash memory metadata and restoring an FTL mapping table, the shadow mapping table and the transaction state table so as to determine the state of the transaction during the two-phase commit. The distributed flash memory transaction processing method disclosed by the invention has the following advantages that the logs are decreased and the transaction processing efficiency under the distributed flash memory environment is improved.

The invention discloses a storage caching method and a storage caching system perceived by a tile recording disk. The method comprises the following implementation steps of: presetting a data block cache and an identification cache; receiving an access request and segmenting the access request into read block requests, if the read block requests hit in the data block cache, returning and reading data from the solid-state disk, otherwise, reading data from the HA-SMR area; directly writing sequential write requests into HA-SMR region; and segmenting the non-sequential write requests into writeblock requests and detecting whether the write block requests hit, and writing the write block requests into the solid-state disk if the write block requests hit. If missed, a write-in position is decided according to the state of the target HA-SMR area, the SMR area is written in the solid-state disk if the HA-SMR area is in a sequential area state, otherwise, the HA-SMR area is written in the tile recording disk and caches the identifier. The host perception characteristic of the HA-SMR disk accurately identifies sequential write requests and harmless non-sequential write requests of the tile recording disk in the cache admission stage, and the requests are directly written into the tile recording disk, so that the write amount of the solid-state disk is reduced.

The invention provides a stream intelligence prediction differencing and compression algorithm, which comprises the following steps: a, determining a source data stream and a new data stream; b, determining the most suitable dictionary according to the new data stream; c, performing matching on the new data stream based on the most suitable dictionary; d, coding the new data stream according to a matching result; e, determining a differencing data stream according to a coding result. The invention also provides a stream intelligence prediction differencing and compression unit, which comprises a first determining device, a second determining device, a first processing unit, a first coding device and a third determining device, wherein the first determining device is used for determining the source data stream and the new data stream; the second determining device is used for determining the most suitable dictionary according to the new data stream; the first processing unit is used for performing the matching on the new data stream based on the most suitable dictionary; the first coding device is used for coding the new data stream according to the matching result; the third determining device is used for determining the differencing data stream according to the coding result. Different from a traditional dictionary compression algorithm adopting one dictionary, the stream intelligence prediction differencing and compression algorithm provided by the invention is capable of generating multiple dictionaries so as to generate the optimal effect. The stream intelligence prediction differencing and compression algorithm has network adaptivity.

The present invention discloses an erasure code storage-based file modification algorithm optimization method. The method comprises the steps of calculating data stored through an erasure code technology by using an algorithm, then splitting the data in a K:M manner, performing object processing on K data blocks and M verification blocks, and then storing in a unit of object; when the stored data is modified, mapping the modified data to the corresponding data block, so as to locate an object name corresponding to data on the block, that is, data modification refers to change of a specific object; when the data object is changed, recording the changed objects by the algorithm so as to lock unchanged data objects, and marking a correspondence; and writing new object data after the modified data is stored. The method has the advantages that the writing amount of the modified stored large file is small and modified data is restored efficiently.

The invention relates to a rapid persistence method based on CAD (computer-aided design), which comprises the following steps: (1) determining the total number N of nodes in a distributed system cluster, letting F = (N-1)/2, setting the number of erasure codes as k, and setting the value of a selection parameter x; (2) judging whether the number of response nodes recently received by the Leader node in the cluster is greater than or equal to F, and if so, executing the step (3); (3) judging whether the number of response nodes recently received by the Leader node in the cluster is greater than or equal to F + k; if the number of response nodes recently received by the Leader node is larger than or equal to F + k, copying is carried out according to an F-CAD consistency protocol until all data are persistently written into F + 1 + x nodes, the log entry is submitted, and next round of copying is executed; and if the number of response nodes received by the Leader node recently is smaller than F + k, performing consistent copying on the system according to a CAD protocol until full data is persistently written into at least F + 1 + 0 nodes, submitting the log entry, and executing the next round of copying. The method can reduce the probability of single-point failure.

The invention discloses a disk controllable buffer writing method, a controller, a hybrid IO scheduling method and a scheduler. The invention belongs to the technical field of computer storage. The method comprises: predicting a write delay state of a next write request according to write delay characteristics of a disk, if a disk request queue is empty and a prediction result is high, setting thedisk state to be unavailable, writing filling data until a next prediction result is not high, resetting ADW to be 0, and resetting the state to be available; if the disk request queue is empty and the prediction result is yes, maintaining that the disk is available and writing filling data; and when the heterogeneous storage node receives the write IO request, if l(t) is greater than L and the write delay state of the next write request in the disk is low or medium, or l(t) is less than or equal to L and the write delay state of the next write request in the disk is low, redirecting the request to the disk. The overall performance of the disk can be improved, the service life of the high-speed hard disk in the heterogeneous storage node can be prolonged, and the disk utilization rate canbe increased.

The invention provides a log generation method based on an FTL table. The method comprises the following steps: recording a logical address and a physical address corresponding to an operation requestas log entries; generating log frames by using the log entries; generating FTL frames by using part of FTL blocks, wherein the FTL table includes multiple FTL blocks; and storing the generated FTL frames and the first log frame on an NVM. The invention also provides a method for reconstructing the FTL table by using logs. The method comprises the following steps: acquiring storage locations of the FTL frames and the log frames of the FTL blocks; reading multiple FTL frames and multiple log frames that belong to the FTL blocks; reconstructing the FTL blocks by using the multiple FTL frames; and updating the FTL blocks by using the log frames, wherein the FTL blocks are part of the FTL table.

The invention provides a message distribution method and device. The message distribution method includes: determining receivers of messages to be distributed; according to the receivers, triggering at least one attention user of the receivers to release the to-be-distributed messages, where the attention users include the receivers, users obtained by registration of tags that the receivers pay attention to, and broadcast users that all the users pay attention to within a preset range; sending the to-be-distributed messages released by the attention users of the receivers to the receivers. By the method, data read-in amount can be reduced, message distribution efficiency is improved, and resources are saved.

The purpose of the present invention is to avoid prolonging processing time involved in a power supply being switched on and off and to extend the useful lifespan of non-volatile memory. Counter information and setting information are stored by an EEPROM (115). The counter information and setting information are stored by a first RAM (116) and a second RAM (117). Storage regions for the counter information and setting information in the first RAM (116) are managed as updatable, and storage regions for the counter information and the setting information of the second RAM (117) are managed as non-updatable, by a memory management unit (118). When a change occurs in the content of the counter information and setting information, the counter information and setting information of the first RAM(116) are overwritten by a system control unit (122) according to the content of the change. When the power supply is switched off, the counter information and setting information of the first RAM (116) and the counter information and setting information of the second RAM (117) are read and compared, and only the information differing therebetween is written to the EEPROM (115).

The invention discloses a digital watermark generation method and system for remote sensing images and a control method therefor. The disclosed generation method comprises the steps of obtaining position relation information between at least two adjacent remote sensing images; setting a digital watermark type; setting digital watermark information; generating a first roulette digital watermark of a left-right digital watermark type on the two remote sensing images in the left-right position relation, and/or, generating a second roulette digital watermark of an up-down digital watermark type on the two remote sensing images in the up-down position relation, and/or, generating a third roulette digital watermark of a four-corner digital watermark type on the four remote sensing images of the four-corner position relationship, wherein the first roulette digital watermark, the second roulette digital watermark and the third roulette digital watermark all comprise digital watermark information. According to the scheme, the problems that the information writing amount is large and the position relation between the remote sensing images cannot be distinguished when digital watermarks are applied to a large batch of remote sensing images with position information can be solved.

The present invention relates to the field of data storage. Disclosed are a data writing method and device for solving the problem of the instable reduction of write delay, the method comprising: sampling data to be written according to a preset sampling algorithm to obtain a target label, the target label being used to represent numerical value distribution of data bits in the data to be written; searching a preset target group label set for a target group label having the highest similarity to the target label, the preset group label set consisting of at least one group label, and each group label corresponding to at least two blank data pages having similar data characteristics; selecting a target blank data page from the at least one blank data page corresponding to the target group label; and writing the data bits in the data to be written that are different from those in the target blank data page into the corresponding positions on the target blank data page. The present invention is mainly used in data read and write operations.

The invention discloses an erasure code storage-based file modification algorithm optimization method, which comprises the following steps of: calculating data stored according to an erasure code technology, splitting the data according to a K: M manner, carrying out objectification processing on K data blocks and M check blocks, and storing the K data blocks and the M check blocks by taking an object as a unit; when the stored data is modified and the data of the modified part is mapped to the corresponding data block, an object name corresponding to the data on the block can be positioned, i.e., the data is modified and a specific object is changed; when the data objects are changed, the changed objects can be recorded by the algorithm, so that the data objects which are not changed canbe locked, and the corresponding relationship among the data objects can be marked; and writing the modified data into new object data after the modified data is stored. The method has the advantagesthat the writing amount of the modified stored large file is small, and the re-storage efficiency of the modified data is high.

The invention discloses a data pair copying method, device and system and a medium, and the method comprises the steps: reading data from first storage equipment, and obtaining first target data; judging whether second target data exists in the first target data or not, wherein the second target data is data of a first preset size and in a preset format, and the preset format is a data format of return data corresponding to a storage space in which data is not written in the first storage device; if the second target data exists in the first target data, filtering the second target data from the first target data to obtain valid data; and writing the valid data into a second storage device. Thus, useless data can be filtered, only valid data is written, the data writing amount is reduced,the copying time is shortened, meanwhile, the garbage collection triggering frequency is reduced, and therefore the user experience is improved.