Data processing method in ordered log mode and computing device

Abstract

The embodiment of the application discloses a data processing method in an ordered log mode and a computing device, and relates to the technical field of servers. The method comprises the following steps: firstly, in response to a first operation performed on a file in an Ext4FS, determining whether the ordered log mode is in a reliable mode; the first operation is an operation of changing the length of file metadata in the Ext4FS. Then, in the case that the ordered log mode is in the reliable mode, writing data corresponding to the metadata in the first storage medium into a second storage medium; the first storage medium is a volatile storage medium; the second storage medium is a non-volatile storage medium; the metadata is used to indicate attribute information of the file. In this way, whenever the metadata changes, the data corresponding to the metadata is written into a storage medium that can realize persistent storage. Since the data has been stored in the hard disk persistently, even if the computing device is powered off abnormally or crashes, the content of the file will not have problems such as garbled code and loss, thereby improving the reliability of the ordered log mode.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of servers, and particularly relates to a data processing method in an Ordered log mode and a computing device. BACKGROUND

[0002] The fourth extended file system (Ext4 FS) is a log file system under a Linux operating system. At present, when a series of operations such as modifying file content, creating and deleting a file are performed on a file in the Ext4 FS, a log area in an Ordered log mode of the Ext4 FS is used to record metadata (a file number, a position, a length, a time, a permission, etc.) and data content of the file. However, the timing of falling disk of the metadata and the data is different. After the metadata has been fallen disk, if the computing device is abnormally powered off or crashes, and the data has not been fallen disk, the content of the file is likely to have garbled code problems, resulting in that the reliability of the Ordered log mode of the Ext4 FS is relatively low. SUMMARY

[0003] Embodiments of the present application provide a data processing method in an Ordered log mode and a computing device, so as to improve the reliability of the Ordered log mode of the Ext4 FS.

[0004] In a first aspect, embodiments of the present application provide a data processing method in an Ordered log mode, which comprises: first, in response to a first operation performed on a file in the Ext4 FS, determining whether the Ordered log mode is in a reliable mode; the first operation herein can be an operation of changing the metadata length of the file in the Ext4 FS; and then, in the case that the Ordered log mode is in the reliable mode, writing data corresponding to the metadata in a first storage medium into a second storage medium; the first storage medium herein is a volatile storage medium, such as a cache; the second storage medium is a non-volatile storage medium, such as a hard disk; and the metadata is used to indicate attribute information of the file, such as a file number, a length, a position, a type, a time and a permission.

[0005] In this way, in the case that the Ordered log mode is in the reliable mode, every time the metadata is changed, the data corresponding to the metadata in the first storage medium is written into the second storage medium which can realize persistent storage, for example, a hard disk. The data corresponding to the file metadata is already completed to be fallen disk before the file save operation is performed. Since the data has been stored in the hard disk in a persistent manner, even if the computing device is abnormally powered off or crashes, the content of the file will not have garbled code and loss problems, the reliability of the Ordered log mode is improved, and the consistency, integrity and security of the data are ensured, and the risk of data loss is reduced.

[0006] In a possible implementation manner of the first aspect, the determining whether the Ordered journal mode is in the reliable mode comprises: determining whether the first parameter in the kernel command line is a first preset value; the first preset value is used to indicate that the state of the reliable mode is enabled; and in a case where the first parameter in the kernel command line is the first preset value, it is determined that the Ordered journal mode is in the reliable mode.

[0007] In another possible implementation manner of the first aspect, the determining whether the Ordered journal mode is in the reliable mode comprises: determining whether the second parameter in the mounting command is a second preset value; the second preset value is used to indicate that the state of the reliable mode is enabled; and in a case where the second parameter in the mounting command is the second preset value, it is determined that the Ordered journal mode is in the reliable mode.

[0008] In another possible implementation manner of the first aspect, the determining whether the Ordered journal mode is in the reliable mode comprises: determining whether the first parameter in the kernel command line is a first preset value and whether the second parameter in the mounting command is a second preset value; the first preset value and the second preset value are both used to indicate that the state of the reliable mode is enabled; and in a case where the first parameter in the kernel command line is the first preset value and the second parameter in the mounting command is the second preset value, it is determined that the Ordered journal mode is in the reliable mode.

[0009] In a case where the preset parameter value in the kernel command line indicates that the state of the reliable mode is enabled, the reliable mode of the Ordered journal mode is globally effective in the entire disk space. In a case where the preset parameter value in the mounting command indicates that the state of the reliable mode is enabled, the reliable mode of the Ordered journal mode is locally effective in part of the disk space. In the reliable mode, when the length of the metadata changes, the disk space can be allocated to the data corresponding to the metadata in the cache as soon as possible without waiting for the allocation delay, and the reliability of the Ordered journal mode is improved.

[0010] In another possible implementation manner of the first aspect, the first operation comprises any one of increasing file content, decreasing file content, creating a file and deleting a file. The use of the file system enables the user to flexibly modify the file content, create and delete files, and the execution of the series of operations facilitates the user to manage the files.

[0011] In another possible implementation manner of the first aspect, before the determining whether the Ordered journal mode is in the reliable mode in response to the first operation performed on the file in the Ext4 FS, the method further comprises: determining whether the file on which the first operation is performed is a preset important level file; and in a case where the file on which the first operation is performed is the preset important level file, enabling the reliable mode of the Ordered journal mode.

[0012] In a possible implementation of the first aspect, the method further includes: in response to the first operation performed on the file in the Ext4 FS, determining whether the Ordered Log mode is in the reliable mode; wherein the first operation is an operation of changing a length of metadata of the file in the Ext4 FS.

[0013] According to different importance levels of files in actual demand scenarios, the reliable mode of the Ordered Log mode can be flexibly set to be turned on or turned off. For example, for a file with a high importance level and strict format requirements, the reliable mode is turned on to ensure the integrity and security of data. For a file with a low importance level, non-strict format requirements, or no importance, the reliable mode is turned off to ensure the performance of data writing to a disk.

[0014] In a possible implementation of the first aspect, before the determining whether the Ordered Log mode is in the reliable mode in response to the first operation performed on the file in the Ext4 FS, the method further includes: writing metadata in the first storage medium to a second storage medium. Based on the metadata being written to the second storage medium first in the Ordered Log mode, the persistent storage of the attribute information of the file is implemented.

[0015] In a possible implementation of the first aspect, the method further includes: periodically writing data in the first storage medium to the second storage medium in a case where the Ordered Log mode is not in the reliable mode.

[0016] In a case where the Ordered Log mode is not in the reliable mode, the data in the cache is periodically written to the disk, that is, the data is periodically allocated with disk space in the cache. In this way, in a case where the computing device is not abnormally powered off or crashes, that is, in a case where the computing device is normally operated, the disk space allocation of the file can be optimized, and the writing performance can be improved.

[0017] In a second aspect, an embodiment of the present application provides a data processing apparatus in an Ordered Log mode, which comprises: a determination module configured to determine whether an Ordered Log mode is in a reliable mode in response to a first operation performed on a file in an Ext4 FS; wherein the first operation is an operation of changing a length of metadata of the file in the Ext4 FS. A storage module configured to write data corresponding to the metadata in a first storage medium to a second storage medium in a case where the Ordered Log mode is in the reliable mode; wherein the first storage medium is a volatile storage medium; the second storage medium is a non-volatile storage medium; and the metadata is used to indicate attribute information of the file.

[0018] In a possible implementation manner of the second aspect, when the determining module determines whether the Ordered journal mode is in the reliable mode, specifically, the determining module is configured to: determine whether the first parameter in the kernel command line is a first preset value; the first preset value is used to indicate that the state of the reliable mode is enabled; and in a case where the first parameter in the kernel command line is the first preset value, it is determined that the Ordered journal mode is in the reliable mode.

[0019] In another possible implementation manner of the second aspect, when the determining module determines whether the Ordered journal mode is in the reliable mode, specifically, the determining module is configured to: determine whether the second parameter in the mounting command is a second preset value; the second preset value is used to indicate that the state of the reliable mode is enabled; and in a case where the second parameter in the mounting command is the second preset value, it is determined that the Ordered journal mode is in the reliable mode.

[0020] In another possible implementation manner of the second aspect, when the determining module determines whether the Ordered journal mode is in the reliable mode, specifically, the determining module is configured to: determine whether the first parameter in the kernel command line is a first preset value and whether the second parameter in the mounting command is a second preset value; the first preset value and the second preset value are both used to indicate that the state of the reliable mode is enabled; and in a case where the first parameter in the kernel command line is the first preset value and the second parameter in the mounting command is the second preset value, it is determined that the Ordered journal mode is in the reliable mode.

[0021] In another possible implementation manner of the second aspect, the first operation includes any one of increasing file content, decreasing file content, creating a file, and deleting a file.

[0022] In another possible implementation manner of the second aspect, before the determining module determines whether the Ordered journal mode is in the reliable mode in response to the first operation performed on the file in the Ext4 FS, the determining module is further configured to: determine whether the file on which the first operation is performed is a preset important level file; and in a case where the file on which the first operation is performed is the preset important level file, the reliable mode of the Ordered journal mode is enabled.

[0023] In another possible implementation manner of the second aspect, the determining module is further configured to: in a case where the file on which the first operation is performed is a non-preset important level file, the reliable mode of the Ordered journal mode is disabled.

[0024] In another possible implementation manner of the second aspect, before the determining module determines whether the Ordered journal mode is in the reliable mode in response to the first operation performed on the file in the Ext4 FS, the storage module is further configured to: write the metadata in the first storage medium to a second storage medium.

[0025] In a further possible implementation form of the second aspect, the storage module is further configured to periodically write the data in the first storage medium to the second storage medium in case the Ordered Log mode is not in the reliable mode.

[0026] In a further possible implementation form of the second aspect, the first storage medium is a cache and the second storage medium is a hard disk.

[0027] In a third aspect, a computing device is provided, comprising a memory configured to store a set of computer-executable instructions, and a processor, wherein when the processor executes the set of computer-executable instructions as the execution device in the first aspect or any possible implementation form of the first aspect, the data processing method in the Ordered Log mode in the first aspect or any possible implementation form of the first aspect is performed.

[0028] In a fourth aspect, a chip is provided, comprising an interface circuit configured to receive code instructions and transmit the code instructions to a processor, and the processor configured to execute the code instructions to perform the data processing method in the Ordered Log mode in the first aspect or any possible implementation form of the first aspect.

[0029] In a fifth aspect, a computer-readable storage medium is provided, storing computer-executable instructions, which, when executed on a computing device, cause the computing device to perform the data processing method in the Ordered Log mode in the first aspect or any possible implementation form of the first aspect.

[0030] In a sixth aspect, a computer program product is provided, comprising computer-executable instructions, which, when executed on a computing device, cause the computing device to perform the data processing method in the Ordered Log mode in the first aspect or any possible implementation form of the first aspect.

[0031] The technical effects brought by any implementation form of the second aspect to the sixth aspect can be referred to the technical effects brought by different implementation forms of the first aspect, which will not be repeated here.

[0032] On the basis of the implementation forms of the above aspects provided by the present application, further combinations can be made to provide more implementation forms. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 A schematic diagram of different log modes of an Ext4 FS provided by an embodiment of the present application;

[0034] Figure 2 A structural schematic diagram of a computing device provided by an embodiment of the present application;

[0035] Figure 3 A flowchart illustrating a data processing method in Ordered log mode provided in an embodiment of this application;

[0036] Figure 4 A flowchart illustrating another data processing method in Ordered log mode provided in an embodiment of this application;

[0037] Figure 5 This is a schematic diagram of a data processing device in Ordered log mode provided in an embodiment of this application. Detailed Implementation

[0038] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0039] In the description of this application, unless otherwise stated, "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple. Additionally, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or order of execution, and that "first," "second," etc., are not necessarily different. Furthermore, in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate that something is being used as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner for ease of understanding.

[0040] To ensure clarity and conciseness in the description of the following embodiments, the terminology used in this application will be briefly introduced first.

[0041] A file system (FS) is a way of organizing and managing files in persistent storage devices (such as hard disks, U disks, and optical drives) by a computer operating system, responsible for file storage, and can implement protection and retrieval of stored files. By using a file system, files can be flexibly created, deleted, modified, and queried, and files can be managed according to a hierarchical file organization structure, such as an ordered file organization structure that includes multiple levels of directories and multiple files or folders under each directory.

[0042] Different file systems organize and manage files differently. For example, the extended file system (Ext FS) under the Linux operating system uses an index node to manage files and records file attribute information and content information. The network file system (NFS) is a distributed file system based on a network file sharing protocol, which can enable different computing devices to jointly manage files over a network. The new generation file system (X filesystem, XFS) is a distributed file system based on a peer-to-peer (P2P) network protocol, which combines encryption technology to manage files and provides secure storage services for individual users and enterprise users.

[0043] Among them, the Ext FS can be a widely used file system under the Linux operating system. Ext4 FS, as the latest member of the extended file system, is the default file system for most Linux operating system versions, which can provide new functions such as larger file capacity, more directory quantity, and finer timestamp unit, has higher performance and good reliability, and is widely used in personal computers, large servers, embedded systems, and big data computing environments.

[0044] Ext4 FS, as the latest version of the log file system, can implement the log recording function of Ext4 FS by the second generation of journaling block devices (journaling block device 2, JDB2). The log area created by JDB2 can record a series of operations performed on files in the file system, such as creating, modifying, deleting, and querying files. Based on this, when a computing device crashes due to abnormal power failure or downtime, after restarting the computing device, the information recorded in the log area can be used to restore the file system to the state before the computing device restarts, to maintain the synchronization of the file system and ensure the consistency of file data.

[0045] Ext4 FS usually has three different log modes, Figure 1 A schematic diagram of different log modes of Ext4 FS provided by an embodiment of the present application. As shown in Figure 1As shown, the log modes of the Ext4 FS include a Journal log mode, an Ordered log mode, and a Writeback log mode.

[0046] In the Journal log mode, the metadata (attribute information such as the number, length, position, type, time, and permission of a file) and data (content information contained in the file) of the file are recorded in the log area at the same time, and both the metadata and the data of the file are written to the disk, which maximizes the guarantee that no data loss occurs when the computing device crashes. This mode has high reliability, but recording the log occupies a large overhead and affects the performance, that is, the performance is low.

[0047] In the Ordered log mode, only the metadata of the file is recorded in the log area, the metadata of the file is first written to the disk, and the data of the file is then sequentially written to the disk. The metadata of the file is protected when the computing device crashes, and this mode has good reliability and performance.

[0048] In the Writeback log mode, only the metadata of the file is recorded in the log area, but the order in which the metadata and the data are written to the disk cannot be guaranteed, and both are written to the disk in an order that is beneficial to the best performance. When the computing device crashes, part of the data of the file may be lost or damaged, and this mode has high performance but low reliability.

[0049] Because the Ordered log mode has good reliability and performance, it is the default log mode of the Ext4 FS. Currently, when a series of functions such as file creation, modification, and deletion are implemented, the log area in the Ordered log mode can be used to record the metadata of the file, and the data of the file is sequentially written to the disk. However, the timing of writing the metadata and the data to the disk is different, and after the metadata has been written to the disk, if the computing device abnormally powers off or crashes, and the data has not been written to the disk, the content of the file may have garbled problems, which leads to low reliability of the Ordered log mode of the Ext4 FS.

[0050] To improve the reliability of the Ordered log mode of the Ext4 FS, an embodiment of the present application provides a data processing method in the Ordered log mode, which includes: first, in response to a first operation performed on a file in the Ext4 FS, determining whether the Ordered log mode is in a reliable mode; the first operation can be an operation of changing the metadata length of the file in the Ext4 FS; and then, in the case where the Ordered log mode is in the reliable mode, writing the data corresponding to the metadata in the first storage medium to the second storage medium; the first storage medium is a volatile storage medium, such as a cache, and the second storage medium is a non-volatile storage medium, such as a hard disk; and the metadata is used to indicate attribute information such as the number, length, position, type, time, and permission of the file.

[0051] Thus, in the case that the Ordered log mode is in the reliable mode, whenever the metadata changes, the data corresponding to the metadata in the first storage medium is written into the second storage medium that can realize persistent storage, such as a hard disk. The data corresponding to the file metadata is already completed to be written to the disk before the file save operation is performed. Since the data has been stored persistently in the hard disk, even if the computing device is abnormally powered off or crashes, the content of the file will not have problems such as garbled code and loss, which improves the reliability of the Ordered log mode, and also guarantees the consistency, integrity and security of the data, and reduces the risk of data loss.

[0052] The embodiments of the data processing method in the Ordered log mode provided by the present application will be described in detail below with reference to the accompanying drawings.

[0053] Figure 2 A structural schematic diagram of a computing device provided by an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the computing device 200 includes a processor 210, a buffer cache 220 and a storage device 230. Figure 2

[0054] The processor 210 is the operation center and control center of the computing device 200 for implementing the data processing method in the Ordered log mode, and can be a central processing unit (CPU). Application software and system software can be run on the processor 210. Optionally, the computing device 200 can also include a data processing unit (DPU) and a neural processing unit (NPU) and other computing processing units with computing capabilities, which can be used for data analysis and data processing of the data content of the file in a big data scenario.

[0055] The application software can be a collection of application programs (applications) written by users according to requirements using various programming languages. The system software can be a collection of various programs that control and coordinate various hardware devices in the computing device 200 to realize their functions, and support the development of application software, and provide a running environment for the application software. The most basic system software can be an operating system (OS). In the embodiments of the present application, a Linux operating system is taken as an example to introduce the processor 210 implementing the data processing method in the Ordered log mode under the Linux operating system, and the specific implementation manner can refer to the description of S301-S302 in the following embodiments.​

[0056] As shown in the Linux operating system, the functions that can be implemented under the Linux operating system include system calls and a virtual file system (VFS). Figure 2

[0057] The system call can be a programming interface provided by the Linux operating system to an application program, allowing the application program to interact and communicate with the Linux operating system kernel. The application program can request the Linux operating system to call the corresponding kernel function to process the file operation related task through the system call. The standard file class system call includes read, write, open, and close, and the corresponding kernel function includes sys_read, sys_write, sys_open, and sys_close.

[0058] The VFS is an important software component of the Linux operating system, which can enable different specific file systems (such as Ext2 FS, Ext 4 FS, XFS, and NFS in Linux) to interact with the Linux operating system kernel in a unified manner, and provide a unified and general interface for the application program to access the specific file system through the system call, so as to facilitate user access and management of files. The VFS can make the system call not consider the type of file system and the difference of storage device. Figure 2

[0059] The buffer cache 220 is a storage area introduced by the Linux operating system between the file system and the device driver for caching data, which can be in the random access memory (RAM). When the processor 210 reads the data of a file from the storage device 230, it first tries to read from the buffer cache 220. If the data reading is successful, it is not necessary to access the slow storage device 230 again; if the data reading fails, it is necessary to read the data of the file from the storage device 230 and cache the read data in the buffer cache 220 for the next time to quickly access the data of this file. The buffer cache 220 can also cache data that has not been written into the storage device 230.

[0060] ​​A device driver can also run on the processor 210. The device driver can be an interface for the Linux operating system to interact with and communicate with a hardware device such as the storage device 230, through which the Linux operating system controls the operation of the hardware device. If the device driver corresponding to the hardware device is not installed correctly, the hardware device cannot work normally.

[0061] The storage device 230 is used to persistently store the metadata of the file recorded by the log area in the Ordered log mode, and the data content of the file. The storage device 230 can be a non-volatile storage medium such as a hard disk, a U disk, and an optical drive. The hard disk here can be a solid state drive (SSD) and a mechanical hard disk drive (HDD) built into the computing device, or a mobile hard drive (MHD) connected externally to the computing device.

[0062] Figure 2 The device structure shown in the figure does not constitute a limitation on the computing device, and can include more or fewer components than the figure, or combine certain components, or different component arrangements. For example, the computing device 200 can also include a memory, a baseboard management controller, a Raid card, and a graphics processing unit (GPU), etc.

[0063] Next, the data processing method in the Ordered log mode provided by the embodiments of the present application will be described. Figures 3 to 4 The data processing method in the Ordered log mode provided by the embodiments of the present application will be described. Figure 3 A flowchart of a data processing method in the Ordered log mode provided by the embodiments of the present application. The method flow is executed by a computing device, specifically by a processor in the computing device. The method can include the following steps.

[0064] S301, in response to a first operation performed on a file in the Ext4 FS, determining whether the Ordered log mode is in a reliable mode.

[0065] The file under the Linux operating system includes two parts of metadata and data. The metadata can be used to indicate the number, length, position, time, and permission of the file, and other attribute information of the file. The data represents the content information contained in the file.

[0066] Generally, an index node is used to store metadata, and a data block is used to store data. The index node includes an i_number field, an i_size field, an i_location field, an i_time field, an i_type field, an i_root field, and the like. The i_time field can include an i_ctime field, an i_mtime field, an i_dtime field, and the like.

[0067] Based on the Ext4 FS, a file can be created, deleted, or modified. The implementation of these functions involves performing a first operation on the file, and the processor can respond to the first operation performed on the file in the Ext4 FS. The first operation can be an operation that changes the length of the file metadata in the Ext4 FS, and can include any one of increasing the file content, decreasing the file content, creating a file, and deleting a file. These operations can involve changes in the i_size field of the metadata.

[0068] For example, new data is written into the file, and the file content is increased, so the length of the file metadata is increased, and the value of the i_size field of the metadata is correspondingly increased.

[0069] For another example, the original data in the file is truncated, and the file content is decreased, so the length of the file metadata is decreased, and the value of the i_size field of the metadata is correspondingly decreased.

[0070] For another example, a new file is created and new data is written into the new file, and the file content is increased, so the length of the file metadata is increased from zero, and the value of the i_size field of the metadata is correspondingly increased from zero.

[0071] For another example, the entire file is deleted, and the data in the entire file is emptied, and the file content is deleted, so the length of the file metadata is zero, and the value of the i_size field of the metadata is correspondingly zero.

[0072] Optionally, in the case of increasing or decreasing the file content, the i_mtime field of the metadata can record the time when the file content is changed. In the case of creating the file content, the i_ctime field of the metadata can record the time when the new file is created. In the case of deleting the file content, the i_dtime field of the metadata can record the time when the file is deleted.

[0073] No matter whether the file content is increased or decreased, or the file is created or deleted, the i_number field of the metadata can be used as the unique identifier of the file. The i_type field of the metadata can record the type of the file, such as a document, a table, a presentation, etc. The i_location field of the metadata can record the path of the file in the hard disk space. The i_root field of the metadata can record the read and write permissions of the file.

[0074] After the processor responds to the operation of changing the metadata length in the Ext4 FS, it also needs to determine whether the Ordered log mode is in the reliable mode. In order to write the data corresponding to the metadata into the non-volatile storage medium when the metadata length is changed in the reliable mode.

[0075] In some embodiments, the processor can determine whether the Ordered log mode is in the reliable mode by judging whether the state of the reliable mode is turned on in the kernel command line or in the mounting command. The following lists three different implementation manners to determine the reliable mode.

[0076] Manner 1: In the case that the first parameter in the kernel command line is a first preset value, it is determined that the Ordered log mode is in the reliable mode.

[0077] The processor can pre-set the first preset value of the first parameter in the kernel command line. The first preset value here can be used to indicate that the state of the reliable mode is turned on when the Linux operating system kernel is started, that is, the reliable mode is turned on after the computing device is powered on.

[0078] The processor first determines whether the first parameter in the kernel command line is the first preset value; and in the case that the first parameter in the kernel command line is the first preset value, it is determined that the Ordered log mode is in the reliable mode.

[0079] For example, the first parameter can include ext4.speed, and the first preset value can include ext4.speed=0. In the case that the ext4.speed takes the value of 0 in the kernel command line (cmdline) parameter sequence for starting the Linux operating system kernel, the processor can determine that the Ordered log mode is in the reliable mode. In this reliable mode, when the metadata length is changed, the data corresponding to the metadata in the cache can be allocated disk space as soon as possible without waiting for the allocation delay.

[0080] Manner 2: In the case that the second parameter in the mounting command is a second preset value, it is determined that the Ordered log mode is in the reliable mode.

[0081] The processor can preset the second preset value of the second parameter in the mount command, and the second preset value can be used to indicate that the state of the reliable mode is turned on when the file is mounted, i.e., the reliable mode is turned on after the file is mounted to the specified mount directory.

[0082] The processor first determines whether the second parameter in the mount command is the second preset value, and then determines that the Ordered log mode is in the reliable mode when the second parameter in the mount command is the second preset value.

[0083] For example, the second parameter can include a mount command, and the second preset value can include "-o nospeed". The preset value "-o nospeed" is added to the basic format of the mount command that can realize file mounting, and the preset value is located between the disk partition directory and the mount directory in the mount command, i.e., mount / disk partition directory / -o nospeed / mount directory. When the mount-o nospeed exists in the mount command, the processor can determine that the Ordered log mode is in the reliable mode. In this reliable mode, when the metadata length changes, the data corresponding to the metadata in the cache can be allocated to the disk space under the specified disk partition directory as soon as possible without waiting for the allocation delay.

[0084] For example, if the disk partition directory is dev / sdb1 and the mount directory is mnt, the parameter configuration of the mount command is mount / dev / sdb1 / -o nospeed / mnt, which indicates that the files belonging to the Ext4 FS in the dev / sdb1 disk partition directory are mounted to the specified mnt mount directory. When the metadata length changes, the data corresponding to the metadata in the cache can be allocated to the disk space under the dev / sdb1 disk partition directory as soon as possible without waiting for the allocation delay.

[0085] Mode 3: In the case that the first parameter in the kernel command line is the first preset value and the second parameter in the mount command is the second preset value, it is determined that the Ordered log mode is in the reliable mode.

[0086] Mode 3 combines the implementation methods of determining the reliable mode in Mode 1 and Mode 2 described above. The processor first determines whether the first parameter in the kernel command line is the first preset value and whether the second parameter in the mount command is the second preset value. In the case that the first parameter in the kernel command line is the first preset value and the second parameter in the mount command is the second preset value, it is also determined that the Ordered log mode is in the reliable mode.

[0087] In some embodiments, when the state of the reliable mode is indicated as on in both the kernel command line and the mounting command, the reliable mode globally takes effect instead of locally, making the reliable mode applicable to files in the Ext4 FS under the entire disk space instead of the Ext4 FS under a certain disk partition, according to the configuration of the kernel command line.

[0088] After the processor determines that the Ordered Log mode is in the reliable mode, the data corresponding to the metadata needs to be written into the storage medium that can be persistently stored at a suitable time. The specific implementation manner can refer to the content set forth in S302 below.

[0089] S302, in the case where the Ordered Log mode is in the reliable mode, write the data corresponding to the metadata in the first storage medium into the second storage medium.

[0090] Based on the fact that, in the Ordered Log mode, the metadata is first written to the disk, before the processor performs the first operation on the file in the Ext4 FS, the metadata in the first storage medium can be written into the second storage medium, so that the metadata of the file is recorded in the log area in the Ordered Log mode.

[0091] The first storage medium is a volatile storage medium, which can be a storage medium such as a cache, for example, a cache memory, and is used to temporarily store metadata. The second storage medium is a non-volatile storage medium, which can be a storage medium such as a hard disk, for example, a solid state disk, a mechanical hard disk, and a hybrid hard disk (a combination of a solid state disk and a mechanical hard disk), and is used to persistently store metadata.

[0092] In the case where the Ordered Log mode is in the reliable mode, when the processor detects that the metadata in the first storage medium (such as a cache) changes, the data corresponding to the metadata in the first storage medium is written into the second storage medium (such as a hard disk), so that the data is written to the disk as soon as possible instead of being periodically written to the disk, that is, the data is no longer allocated disk space with delay.

[0093] In some embodiments, the processor writes the data corresponding to the metadata into the second storage medium when detecting that the length field of the metadata in the first storage medium is changed. For example, when the processor detects that the length field i_size of the metadata in the first storage medium is changed, the ext4_update_inode_size function is called to return a Boolean value such as i_size_changed==true or i_size_changed==1, indicating that the length field i_size is changed. The processor adds the data corresponding to the metadata to the ordered data list of the log area created by the JDB2 in the Ordered log mode, so as to write the data corresponding to the metadata into the hard disk, thereby completing the data writing.

[0094] Since the length of the file metadata is changed, the data corresponding to the metadata is written to the hard disk. The data is stored in the hard disk in a persistent manner. Even if the computing device is powered off or crashes abnormally before the file saving operation is performed, the content of the file will not be garbled or lost, thereby improving the reliability of the Ordered log mode of the Ext4 FS, ensuring the integrity and security of the data, and reducing the risk of data loss.

[0095] The above embodiments introduce the implementation of the processor determining that the Ordered log mode is in the reliable mode, and the method of persistently storing the data corresponding to the metadata in the reliable mode.

[0096] In some other embodiments, the processor can periodically write the data in the first storage medium into the second storage medium when determining that the Ordered log mode is not in the reliable mode. The data in the cache is periodically written to the hard disk, i.e., the data in the cache is periodically allocated with disk space. In this way, the disk space allocation of the file can be optimized and the writing performance can be improved in the normal operation state of the computing device.

[0097] For example, when the Linux operating system kernel is started or the file is mounted, the state of the reliable mode is closed, and the Ordered log mode is not in the reliable mode.

[0098] For example, the first preset value of the first parameter in the kernel command line is adjusted from “ext4.speed=0” to “ext4.speed=1”, which indicates that the state of the reliable mode is closed in the kernel command line for starting the Linux operating system kernel. The processor can determine that the Ordered log mode is not in the reliable mode.

[0099] For example, the second preset value of the second parameter in the mounting command is adjusted from -o nospeed to -o speed, and the format of the corresponding mount command is mount / disk partition directory / -o speed / mounting directory. In this way, the state of indicating the reliable mode in the file mounting mount command is closed, and the processor can determine that the Ordered log mode is not in the reliable mode.

[0100] According to the data processing method in the Ordered log mode provided in the embodiments of the present application, the user can flexibly set the reliable mode of the Ordered log mode according to the importance of the file in the actual demand scenario.

[0101] In some embodiments, before determining whether the Ordered log mode is in the reliable mode in response to the first operation performed on the Ext4 FS, the processor can first determine whether the file to be executed by the first operation is a preset important level file, and the preset important level is used to indicate the importance of the file.

[0102] In the case that the file to be executed by the first operation is a preset important level file, the reliable mode of the Ordered log mode is enabled. For the file with high importance, strict format requirement or need for confidentiality, enabling the reliable mode can ensure the integrity and security of the data.

[0103] In the case that the file to be executed by the first operation is a non-pre-set important level file, the reliable mode of the Ordered log mode is disabled. For the file with low importance and non-strict format requirement, disabling the reliable mode can ensure the performance of data writing to the disk.

[0104] The flow of the data processing method in the Ordered log mode provided in the above embodiments will be described below with reference to the accompanying drawings. Figure 4 Another flowchart of the data processing method in the Ordered log mode provided in the embodiments of the present application is shown in FIG. 4. As shown in FIG. 4, the method performed by the processor includes the following steps. Figure 4

[0105] S40, set the working mode of the Ordered log mode. The working mode includes enabling the reliable mode of the Ordered log mode and disabling the reliable mode of the Ordered log mode.

[0106] S41, write the metadata in the first storage medium into the second storage medium.

[0107] S42, perform the first operation on the file in the Ext4 FS.

[0108] ​S43, determine whether the Ordered log mode is in the reliable mode. The method described in S301 can be referred to determine whether the reliable mode is on.

[0109] In the case that the Ordered log mode is in the reliable mode, the reliable mode is on, then the following S44 is executed.

[0110] In the case that the Ordered log mode is not in the reliable mode, the reliable mode is off, then the following S45 is executed.

[0111] S44, in the case that the Ordered log mode is in the reliable mode, write the data corresponding to the metadata in the first storage medium into the second storage medium. The method described in S302 can be referred to to realize writing the data corresponding to the metadata into the second storage medium as soon as possible.

[0112] S45, in the case that the Ordered log mode is not in the reliable mode, determine whether the computing device is abnormally powered off or crashes.

[0113] In the case that the computing device is not abnormally powered off or crashes, then the following S46 is executed.

[0114] In the case that the computing device is abnormally powered off or crashes, since the Ordered log mode is not in the reliable mode, after the metadata is written into the second storage medium, if the computing device is abnormally powered off or crashes, however the data corresponding to the metadata has not been written into the second storage medium, then the data content of the file is likely to have garbled problem, resulting in incomplete file.

[0115] S46, in the case that the computing device is not abnormally powered off or crashes (in the state that the computing device is normally running), determine whether there is a write instruction to write the cached data in the first storage medium into the second storage medium.

[0116] In the case that there is a write instruction to write the cached data in the first storage medium into the second storage medium, then the following S47 is executed.

[0117] In the case that there is no write instruction to write the cached data in the first storage medium into the second storage medium, then the following S48 is executed, until the data is written into the second storage medium is completed, the following S47 is executed.

[0118] S47, in the case that there is a write instruction to write the cached data in the first storage medium into the second storage medium, write the cached data in the first storage medium into the second storage medium in response to the write instruction. In the state that the reliable mode is not on and the computing device is normally running, the data in the cache is periodically written into the disk, which can optimize the disk space allocation of the file.

[0119] S48, in the absence of writing the cached data in the first storage medium into the second storage medium, then waiting for the write command. When the subsequent write instruction occurs, step S47 is executed.

[0120] According to the data processing method in the Ordered log mode provided by the above steps, in the case that the Ordered log mode is the reliable mode, the data corresponding to the metadata in the first storage medium is written into the second storage medium that can realize persistent storage, such as a hard disk, whenever the metadata changes. The data corresponding to the file metadata is already completed to be written to the disk before the file save operation is performed. Since the data has been persistently stored in the hard disk, even if the computing device is abnormally powered off or crashes, the content of the file will not have problems such as garbled code and loss, which improves the reliability of the Ordered log mode, and also guarantees the consistency, integrity and security of the data, and reduces the risk of data loss.

[0121] Based on the fact that the data has been persistently stored in the second storage medium in the case that the reliable mode is enabled. After the computing device that is abnormally powered off or crashes is restarted, the complete data content can be obtained in the second storage medium, which brings convenience to the user in managing files, and improves the user experience of using the Ext4 FS.

[0122] The above mainly describes the solutions provided by the embodiments of the present application from the perspective of the method. In order to realize the above functions, the data processing apparatus in the Ordered log mode comprises a hardware structure and / or a software module for executing each function. Those skilled in the art should easily realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is realized in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0123] The embodiments of the present application can divide the functional modules of the data processing apparatus in the Ordered log mode according to the above method. For example, the data processing apparatus in the Ordered log mode can comprise functional modules corresponding to each function division, or two or more functions can be integrated into one processing module. The above integrated module can be realized in the form of hardware or software functional module. It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical function division. Actual implementation can have another division method.

[0124] For example, Figure 5A structural diagram of a data processing apparatus in an Ordered log mode is provided for an embodiment of the present application. The data processing apparatus 500 in the Ordered log mode performs actions through a processor. The data processing apparatus 500 in the Ordered log mode includes a determination module 510 and a storage module 520.

[0125] The determination module 510 is configured to determine whether the Ordered log mode is in a reliable mode in response to a first operation performed on a file in an Ext4 FS. The first operation is an operation of changing a length of file metadata in the Ext4 FS. For example, the determination module 510 is configured to perform the following steps in S301. Figure 3

[0126] The storage module 520 is configured to write data corresponding to the metadata in the first storage medium into a second storage medium in a case where the Ordered log mode is in the reliable mode. The first storage medium is a volatile storage medium. The second storage medium is a non-volatile storage medium. The metadata is used to indicate attribute information of the file. For example, the storage module 520 is configured to perform the following steps in S302. Figure 3

[0127] Optionally, when the determination module 510 determines whether the Ordered log mode is in the reliable mode, the determination module 510 is specifically configured to determine whether a first parameter in a kernel command line is a first preset value. The first preset value is used to indicate that a state of the reliable mode is turned on. In a case where the first parameter in the kernel command line is the first preset value, it is determined that the Ordered log mode is in the reliable mode.

[0128] Optionally, when the determination module 510 determines whether the Ordered log mode is in the reliable mode, the determination module 510 is specifically configured to determine whether a second parameter in a mounting command is a second preset value. The second preset value is used to indicate that the state of the reliable mode is turned on. In a case where the second parameter in the mounting command is the second preset value, it is determined that the Ordered log mode is in the reliable mode.

[0129] Optionally, when the determination module 510 determines whether the Ordered log mode is in the reliable mode, the determination module 510 is specifically configured to determine whether the first parameter in the kernel command line is the first preset value and whether the second parameter in the mounting command is the second preset value. The first preset value and the second preset value are both used to indicate that the state of the reliable mode is turned on. In a case where the first parameter in the kernel command line is the first preset value and the second parameter in the mounting command is the second preset value, it is determined that the Ordered log mode is in the reliable mode.

[0130] Optionally, the first operation includes any one of increasing file content, decreasing file content, creating a file, and deleting a file. ​​

[0131] Optionally, before determining whether the Ordered journal mode is in the reliable mode in response to the first operation performed on the file in the Ext4 FS, the determining module 510 is further configured to: determine whether the file on which the first operation is performed is a preset important level file; and turn on the reliable mode of the Ordered journal mode in a case where the file on which the first operation is performed is the preset important level file.

[0132] Optionally, the determining module 510 is further configured to: turn off the reliable mode of the Ordered journal mode in a case where the file on which the first operation is performed is a non-pre-set important level file.

[0133] Optionally, before determining whether the Ordered journal mode is in the reliable mode in response to the first operation performed on the file in the Ext4 FS, the storage module 520 is further configured to: write the metadata in the first storage medium into the second storage medium.

[0134] Optionally, the storage module 520 is further configured to: periodically write the data in the first storage medium into the second storage medium in a case where the Ordered journal mode is not in the reliable mode.

[0135] Optionally, the first storage medium is a cache, and the second storage medium is a hard disk.

[0136] For a specific description of the above optional mode, refer to the foregoing method embodiments, which will not be described here again. In addition, the above-provided explanation and beneficial effect description of any one of the data processing apparatuses 500 in the Ordered journal mode can refer to the foregoing corresponding method embodiments, which will not be described here again.

[0137] The embodiment of the present application further provides a computing device, which comprises a memory and a processor, the memory stores computer execution instructions, and the processor implements the Ordered journal mode data processing method in the foregoing embodiments when executing the computer execution instructions.

[0138] The embodiment of the present application does not make any limitation on the specific form of the computing device. For example, the computing device can be a physical or logical computing device, or two or more physical or logical computing devices sharing different responsibilities and cooperating with each other to realize the functions of the computing device.

[0139] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and when the computer program runs on a computing device, the computing device executes the method executed by any one of the computing devices provided above.

[0140] The descriptions of the explanations and beneficial effects of the related content in any one of the above provided computer readable storage media can refer to the corresponding embodiments described above, and will not be repeated here.

[0141] The embodiments of the present application further provide a chip. The chip integrates a control circuit and one or more ports for implementing the functions of the above computing device. Optionally, the functions supported by the chip can refer to the above, and will not be repeated here. Those skilled in the art can understand that all or part of the steps of the above embodiments can be instructed by programs to complete the related hardware. The programs can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a random access memory, etc. The processing unit or processor mentioned above can be a central processing unit, a general processor, an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA) or other programmable logic device, a transistor logic device, a hardware component or any combination thereof.

[0142] The embodiments of the present application further provide a computer program product containing instructions, which, when executed on a computing device, cause the computing device to perform any one of the above embodiments. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer device, all or part of the processes or functions according to the embodiments of the present application are generated.

[0143] It should be noted that the above devices for storing computer instructions or computer programs provided by the embodiments of the present application, such as but not limited to the above memory, computer readable storage medium and communication chip, etc., all have non-volatility (non-transitory).

[0144] In the above embodiments, all or part can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed by a computer device, the computer program instructions generate all or part of the processes or functions according to the embodiments of the present application. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or data storage device, such as one or more computer devices, data centers, etc. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0145] Although the present application is described herein in conjunction with various embodiments, it is understood that other variations of the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed application, from an inspection of the drawings, the disclosure, and the appended claims. The word "comprising" does not exclude other components or steps not listed in the claims, and the word "a" or "an" does not exclude a plurality. A single processor or other unit can fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to an advantage.

[0146] Although the present application is described herein in conjunction with specific features and embodiments thereof, it is understood that modifications and combinations can occur to those skilled in the art to which the present application pertains, within its spirit and scope. Accordingly, the description and drawings are to be regarded as illustrative in nature and are not to be taken as limiting the scope of the application as defined in the appended claims. Obviously, various modifications and changes can be made to the present application by those skilled in the art without departing from the spirit and scope of the present application. Accordingly, the present application intends to include all such modifications and changes as fall within the scope of the appended claims and their equivalents.

Claims

1. A data processing method under Ordered log mode, characterized in that, The method includes: Metadata from the first storage medium is written to the second storage medium, wherein the metadata is used to indicate the file's number, length, location, type, time, and permissions; In response to a first operation performed on a file in the fourth-generation extended file system Ext4 FS, it is determined whether the Ordered journaling mode is in reliable mode; wherein the first operation is an operation that changes the length of the file metadata in the Ext4 FS; When the Ordered log mode is in reliable mode, the data corresponding to the metadata in the first storage medium is written to the second storage medium; wherein, the first storage medium is a volatile storage medium; the second storage medium is a non-volatile storage medium; the metadata is used to indicate the attribute information of the file; When the Ordered log mode is not in reliable mode, data in the first storage medium is periodically written to the second storage medium; Determining whether the Ordered log mode is in reliable mode includes: Determine whether the first parameter in the kernel command line is a first preset value; wherein, the first preset value is used to indicate that the reliable mode is enabled; if the first parameter in the kernel command line is the first preset value, determine that the Ordered log mode is in reliable mode; And / or, determine whether the second parameter in the mount command is a second preset value; wherein the second preset value is used to indicate that the reliable mode is enabled; if the second parameter in the mount command is the second preset value, determine that the Ordered log mode is in reliable mode.

2. The method according to claim 1, characterized in that, The first operation includes any one of adding file content, reducing file content, creating a file, or deleting a file.

3. The method according to claim 1 or 2, characterized in that, Before determining whether the Ordered log mode is in reliable mode in response to the first operation performed on a file in the Ext4 FS, the method further includes: Determine whether the file performing the first operation is a file with a preset importance level; If the file performing the first operation is a file with the preset importance level, then enable the reliable mode of the Ordered log mode.

4. The method according to claim 1 or 2, characterized in that, The method further includes: If the file performing the first operation is not a file with a preset importance level, then the reliable mode of the Ordered log mode is turned off.

5. The method according to claim 1 or 2, characterized in that, The first storage medium includes a cache; the second storage medium includes a hard disk.

6. A computing device, characterized in that, The computing device includes a memory and a processor, the memory being used to store computer instructions, and when the processor executes the computer instructions, it performs the method of any one of claims 1-5.

7. A chip, characterized in that, The chip includes an interface circuit and a processor. The interface circuit is used to receive code instructions and transmit them to the processor. When the processor executes the code instructions, it performs the method described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed on a computing device, enable the computing device to perform the method described in any one of claims 1-5.

9. A computer program product, characterized in that, The computer program product includes computer execution instructions, which, when executed on a computing device, enable the computing device to perform the method described in any one of claims 1-5.

Images