13 results about "Journaling file system" patented technology

A system can store a first encryption key for a first data chunk of a log filesystem in a first virtual chunk extent that comprises first virtual pointers that point to the first data chunk, wherein the first data chunk is encrypted with the first encryption key. The system can, based on performing garbage collection on the first data chunk, to produce surviving data that remains from the first data chunk after the garbage collection, move the surviving data to a second data chunk, wherein the surviving data is encrypted in the second data chunk with a second encryption key, wherein the second encryption key is stored in a second virtual chunk extent corresponds to the second data chunk, and delete the first virtual chunk extent while refraining from deleting the first data chunk, wherein, after the deleting, the first encryption key is deleted.

A system can maintain, in a log filesystem, a group of virtual chunk extents arranged in a chain data structure, wherein respective virtual chunk extents of the group of virtual chunk extents comprise respective pointers to respective next virtual chunk extents, and wherein respective virtual pointers of the respective virtual chunk extents identify locations within a data chunk. The system can maintain one backpointers from a chunk descriptor that corresponds to the data chunk, wherein the backpointers points to a head virtual chunk extent of the group of virtual chunk extents. The system can, based on performing an iteration of garbage collection of data in the data chunk, increase a number of virtual pointers of the virtual pointers while refraining from adding any new backpointers, other than the one backpointers, to the chunk descriptor.

A system can store data in a chunk domain of a log filesystem, wherein the chunk domain comprises a group of chunks. The system can, based on determining to perform an operation on the data, wherein the operation comprises modifying the storing of the data from a first redundant array of inexpensive devices configuration to a second redundant array of inexpensive devices configuration, wherein the operation comprises a tiering operation, wherein the operation comprises a re-keying operation, wherein the operation comprises a restriping operation, or wherein the operation comprises a re-encrypting operation, as part of performing forward garbage collection, moving data of the data according to the operation and based on a flexible data placement command that is supported by a storage device that stores the data after the forward garbage collection is performed.

The invention discloses a log file system based on an EEPROM (Electrically Erasable Programmable Read-Only Memory), and particularly relates to the field of storage equipment. The EEPROM comprises a plurality of data blocks; the log file system comprises a file system description area, a file description area and a block storage area; wherein the file system description area is used for storing the number and the size of data blocks in the EEPROM and the number of initial data blocks in the block storage area in the EEPROM, and operating a system description data block bitmap and a file identifier; the file identifier represents the storage state of each preset file in the EEPROM; the file description area is used for operating a file descriptor of each file in an EEPROM (Electrically Erasable Programmable Read-Only Memory); and the block storage area is used for carrying out file operation in the EEPROM. According to the file system, the complete file can be stored in the EEPROM.

This disclosure provides a method, apparatus, device, and storage medium for remote log processing of an application, relating to the field of distributed storage technology. The method includes: obtaining a log processing request from an application; responding to the data processing request using a data cache in a log file system when the log processing request is a data processing request; responding to the metadata processing request using a metadata cache in a log file system when the log processing request is a metadata processing request; and, if the log data in the data cache changes, or if the metadata in the metadata cache changes, flushing the changed log data or changed metadata to a remote distributed file system via an asynchronous worker thread.

A system can create a first, persistent, data structure that indicates a location of an ingested file within a log filesystem. The system can, based on not deduplicating the file at ingest, create a second, temporary, data structure that indicates the location of the file, which comprises a logical backpointer to the file. The system can perform deduplication after creating the second data structure, comprising parsing a chunk descriptor associated with the file to identify the second data structure, identifying any active first virtual mapping structures associated with the file, identifying a deduplicated block associated with the file, updating a leaf mapping block of the file to point to a virtual pointer that identifies data associated with the deduplicated block, based on the logical backpointer of the second data structure, and after processing any second virtual mapping structures of the chunk, freeing the second data structure.

A system can store a first encryption key for a first data chunk of a log filesystem in a first virtual chunk extent that comprises first virtual pointers that point to the first data chunk, wherein the first data chunk is encrypted with the first encryption key. The system can, based on performing garbage collection on the first data chunk, to produce surviving data that remains from the first data chunk after the garbage collection, move the surviving data to a second data chunk, wherein the surviving data is encrypted in the second data chunk with a second encryption key, wherein the second encryption key is stored in a second virtual chunk extent corresponds to the second data chunk, and delete the first virtual chunk extent while refraining from deleting the first data chunk, wherein, after the deleting, the first encryption key is deleted.

The invention relates to a log stream adjustment optimization method, system and device of a log file system, and relates to the technical field of storage system optimizing.The method comprises the steps that load key index information is periodically obtained from a maintained log stream pool for weighting processing, and an adjustment interval of a current period is determined, then log stream dynamic adjustment of load balancing is carried out according to a multi-level adjustment strategy corresponding to the adjustment interval, and when write-back control optimization is carried out, fine-grained division is carried out according to different types of write-back tasks in the constructed initial write-back queue to obtain multiple queues used for carrying out write-back of the write-back tasks; and performing write-back control optimization on each to-be-written-back task in the multiple queues to obtain a log flow adjustment optimization result. According to the method and the device, the log stream is adjusted according to the load change in a self-adaptive mode, and excessive dependence on the same queue lock is reduced through fine-grained refinement and localization of the queue, so that the concurrency efficiency and the adaptability of the log file system in the multi-core environment are improved.

The invention relates to a log management method, system, device and medium, and the method comprises the steps: a system log source generates a log, and inputs the log to a log adaptation layer; sending the log to the log cache region through a log adaptation layer; and temporarily storing the log through a log cache region, and writing the log in the log cache region into a log file system under the condition that a trigger condition is met. According to the technical scheme provided by the invention, the log loss can be avoided while the system performance is ensured under the condition that the original system log scheme of the equipment is not influenced.

A journaling file system may implement remote durable logging. Updates to a file system may be received, and log records describing the updates may be stored in a locally-accessible file system change log. The update may then be acknowledged as committed. The log records may then be sent to be stored in a network-based data store in a remote version of the file system change log. Once it may be determined that the log records are stored in the remote version, storage space for the log records in the local file system change log may be reclaimed. Various types of restoration and duplication techniques may be implemented based on the remote version of the change log to restore a file system at an originating device or to duplicate the file system at a different device.

A log writing method, device, electronic device and storage medium. The method includes: obtaining the file operation of the business program (S110); obtaining a first interface corresponding to the file operation from multiple system interfaces configured by the log file system, wherein the system interface is an interface defined based on the portable operating system interface standard (S120); obtaining a second interface corresponding to the first interface from multiple interfaces configured by the back-end log storage system (S130); calling the second interface to write the log file corresponding to the file operation in the memory to the back-end log storage system (S140). Through the above method, the log file system can call the second interface of the back-end log storage system to directly write the log file in the memory to the back-end log file storage system, so that the log file does not need to be written to the disk, so that the log writing is no longer limited by the writing and reading performance of the local disk, and high-speed writing of the log is achieved.

Embodiments of the present disclosure disclose a log management method and device of multiple processing units, a computer readable storage medium and an electronic device, wherein the method comprises: in response to a preset log reading condition being met, determining log update conditions in a storage area corresponding to each second processing unit from a shared memory; and when there is an update log event in the storage area corresponding to each second processing unit, storing update log information corresponding to the update log event in a log file system corresponding to a first processing unit. Embodiments of the present disclosure enable the first processing unit to extract the log of the second processing unit, and the log information is transmitted by means of memory reading and writing, which has a small hardware performance overhead. In addition, the log information of each second processing unit can be physically isolated, improving the security of information storage. By storing the log information through a file system, various modes of management of the log information can be implemented, facilitating log backtracking.

A system can maintain a log filesystem in which data is stored as respective data chunks, wherein the respective data chunks correspond to respective chunk descriptors, and wherein respective inode leaf blocks, which correspond to respective files that are stored in the respective data chunks, comprise respective first virtual pointers to respective virtual chunk extents that comprise respective second virtual pointer groups to the respective data chunks. The system can, for a data chunk of the respective data chunks, store, in a corresponding chunk descriptor, a maximum temperature value and a minimum temperature value of the data chunk. The system can adjust a data tiering level of at least part of the data of the data chunk based on at least one of the maximum temperature value or the minimum temperature value.