Log writing and synchronization method and device, system and computer storage medium

A technology for synchronizing device and writing position, applied in the direction of calculation, input/output to record carrier, and input/output process of data processing, etc., can solve the problem of log disk partition saturation, the impact of solid-state disk performance and life, and the inability to balance log disks Partition storage space and other issues

Active Publication Date: 2018-07-27
ZTE CORP
View PDF3 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] Problem 1: Poor system scalability
The log disk partition leads to poor system scalability, and the number of partitions is difficult to increase or decrease at will, so the system scale cannot be expanded
[0007] Problem 2: Unbalanced storage space
The busyness of each data disk is different, which makes it impossible to balance the storage space of each log disk partition, resulting in waste of individual log disk partition space and saturation of individual log disk partitions
[0008] Problem 3: Damage to SSD lifespan
The old log technology uses an in-place update method when writing. This method does not take into account the erase-before-write characteristics of solid-state drives, so it has a relatively large negative impact on the life of

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Log writing and synchronization method and device, system and computer storage medium
  • Log writing and synchronization method and device, system and computer storage medium
  • Log writing and synchronization method and device, system and computer storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0139] The processing steps of the log writing process part are as follows:

[0140] Step 1: Read CP, LP. The initial value is 0.

[0141] Step 2: Write the log page data into the log data page numbered LP.

[0142] Step 3: Obtain the metadata of the write operation from the file system, including layout, ino, etc. Store layout and ino in the log bitmap area, and the slot number is LP.

[0143] Step 4: LP++.

[0144] Step 5: Determine whether LP is equal to MP, if so, assign LP to SP.

[0145] Step 6: Determine whether LP+5 is equal to CP, and if so, trigger log synchronization.

[0146] The processing steps of the log synchronization process part are as follows:

[0147] Step 1: Read CP, LP.

[0148] Step 2: Check if CP and LP are equal, if they are equal, end. If CP is smaller than LP, go to step 3. If CP is greater than LP, go to step 7.

[0149] Step 3: Read the log bitmap area, and traverse the metadata layout of slot numbers from CP to LP for duplication. If t...

Embodiment 2

[0159] The processing steps of the log writing process part are as follows:

[0160] Step 1: Read CP, LP.

[0161] Step 2: Get the metadata of the write operation from the file system, including layout, BlockID, etc. Store the layout and BlockID in the log bitmap area, and the slot number is LP.

[0162] Step 3: Write the log page data into the log data page numbered LP.

[0163] Step 4: LP++.

[0164] Step 5: Determine whether LP is equal to MP, if so, assign LP to SP.

[0165] Step 6: Determine whether LP+1 is equal to CP, and if so, trigger log synchronization.

[0166] The processing steps of the log synchronization process part are as follows:

[0167] Step 1: Read CP, LP.

[0168] Step 2: Check if CP and LP are equal, if they are equal, end. If CP is smaller than LP, go to step 3. If CP is greater than LP, go to step 6.

[0169] Step 3: Read the log bitmap area, traverse the metadata of slot numbers from CP to LP, and write the content of the corresponding log p...

Embodiment 3

[0177] The application scenario of this embodiment is an Internet small picture storage service, and pictures smaller than 1 KB are defined as small pictures. Therefore, a single page must be able to aggregate and store 4 small pictures. In the bitmap area, an array named gather is added to each slot, the length of the array is 4, and the elements of the array are used to store oids. Among them, the first element is used to store the oid of the small picture stored in the first 1KB of the page, the second element is used to store the oid of the small picture stored in the second 1KB of the page, and so on. When the array element is 0, it means that the 1KB position in the page is vacant.

[0178] The processing steps of the log writing process part are as follows:

[0179] Step 1: Determine whether the file size is less than 1KB, if not, write directly to the disk without writing the log.

[0180] Step 2: Read CP, LP. The initial value is 0.

[0181]Step 3: Read the layou...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a log writing method and device, a system and a computer storage medium. The method comprises the steps that an end point page number which is written into a log is read from avariable area of a log disk; the log is written into a data area page corresponding to the end point page number to form a log page; the writing position corresponding to the log page and a data section number which the log page belongs to are acquired and stored into a bitmap area of the log disk; in the variable area of the log disk, one is added to the end point page number which is written into a log, and whether or not the log is written to a tail disk of the log disk is judged; if yes, an end point which is written into the log is set to a disk head.

Description

technical field [0001] The invention relates to the field of computer storage, in particular to a method, device, system, and computer storage medium for log writing and synchronization based on solid-state hard disks. Background technique [0002] In storage systems, high reliability, high performance, and low cost are the most important and contradictory three major indicators. Journal technology is an effective means to ensure high reliability and high performance of the system. Many storage systems have introduced log technology, such as EXT3, EXT4, XFS, JFS, F2FS, NTFS, Btrfs, CEPH, etc. [0003] In the log technology, the operation of the storage system on data (including creation, addition, modification, deletion, etc.) is first encapsulated into a log and recorded in the log disk. At regular intervals (or when the log disk is about to be filled), the storage system will actually write the data content to the data disk according to the log. This process is called lo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F3/06
CPCG06F3/0608G06F3/0616G06F3/0641G06F3/0644G06F3/0652G06F3/0676
Inventor 杨洪章罗圣美江滢
Owner ZTE CORP
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap