Data storage method and computing device

Abstract

The invention provides a data storage method and computing equipment. The method comprises the steps that a target storage area in the storage equipment is pre-allocated to a file according to a pre-allocation request of an application program; and instructing a controller of the storage device to update the value of the management identifier corresponding to the target storage area to a first target value indicating that the data is not written into the target storage area. According to the method, the controller of the storage equipment modifies the management identifier of the pre-allocated target storage area to erase zero of the storage area, so that the occupied time of the IO interface in the pre-allocation process can be greatly shortened, and the jamming of the computing equipment caused by blocking other IO requests can be avoided.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data storage method and a computing device. Background Art

[0002] In a computing device, when the storage area of ​​an application file is insufficient, a storage area in the storage device needs to be pre-allocated for the file, so as to ensure that the data corresponding to the file is continuous on the storage device. In addition, in order to prevent the application from reading dirty data in the pre-allocated target storage area, the storage area needs to be erased before writing data to the pre-allocated target storage area.

[0003] In the related art, the storage area is erased by writing 0 to the pre-allocated target storage area. That is to say, to erase the storage area, the input / output (IO) interface is called to write the actual data to the storage area, and the written data are all 0. This erasing method takes a lot of time, resulting in the IO interface being occupied for a long time. In the case of pre-allocating storage areas for multiple files, the IO interface being occupied for a long time will cause other IO requests to be blocked, which will cause the computing device to freeze and affect the user experience. Summary of the invention

[0004] The present application provides a data storage method and a computing device, which can significantly reduce the time that an IO interface is occupied during the pre-allocation process by modifying the management identifier of a pre-allocated target storage area to erase the storage area.

[0005] In a first aspect, the present application provides a data storage method. The method can be applied to a computing device, and the method includes: receiving a pre-allocation request from an application, wherein the pre-allocation request includes a file identifier; pre-allocating a target storage area in a storage device for a file corresponding to the file identifier; and instructing a controller of the storage device to update the value of a management identifier corresponding to the target storage area to a first target value, wherein the first target value indicates that no data has been written to the target storage area.

[0006] In the above solution, during the pre-allocation process, the management identifier corresponding to the target storage area is modified by the controller of the storage device, and the target storage area is erased to zero, which can reduce the time occupied by the storage device IO interface during the pre-allocation process.

[0007] In a possible implementation of the first aspect, the method also includes: in a case where it is determined that the target data corresponding to the file needs to be written to the storage device, if the size of the target data is greater than a third target value, writing the target data to the target storage area N times, wherein the size of the data written to the target storage area each time is less than or equal to the third target value.

[0008] In the above scheme, when the target data exceeds the third target value, the target data is written to the storage device in multiple times. By controlling the amount of data written to the storage device each time, the time occupied by the IO interface of the storage device during a single write can be reduced, thereby avoiding the situation where the IO interface is occupied for a long time due to a large amount of written data, causing other IO requests to be blocked and causing the computing device to freeze.

[0009] In a possible implementation of the first aspect, the method further includes: when writing the target data to the target storage area, the controller of the storage device updates the value of the management identifier of the target storage area to a second target value, and the second target value indicates that data has been written to the target storage area.

[0010] In the above solution, after the data is written into the storage device, the controller modifies the management identifier of the target storage area, which can reduce the number of calls to the IO interface of the storage device during the data writing process.

[0011] In a possible implementation of the first aspect, the target storage area includes at least one storage block, and pre-allocating the target storage area in the storage device for the file corresponding to the file identifier includes: adding metadata of one or more data blocks to the metadata corresponding to the file, the metadata of each data block includes a physical address of each data block, and the physical address of each data block maps one or more storage blocks.

[0012] In the above solution, the target storage area is associated with the file through metadata, which makes it convenient to read / write the file data to the target storage area.

[0013] In a possible implementation of the first aspect, the storage device includes a hard disk, and the storage block includes sectors.

[0014] In the above scheme, the pre-allocation process described in the first aspect can be implemented on the hard disk, reducing the time occupied by the pre-allocation process on the hard disk's IO interface and reducing the number of calls to the hard disk's IO interface when writing data to the hard disk.

[0015] In a second aspect, the present application further provides a data storage device. The data storage device can be applied to a computing device, and the data storage device includes: a receiving module and a pre-allocation module.

[0016] The receiving module is used to receive a pre-allocation request from an application program, wherein the pre-allocation request includes a file identifier.

[0017] Among them, the pre-allocation module is used to pre-allocate a target storage area in a storage device for the file corresponding to the file identifier, and to instruct the controller of the storage device to update the value of the management identifier corresponding to the target storage area to a first target value, wherein the first target value indicates that no data has been written to the target storage area.

[0018] In a possible implementation of the second aspect, the data storage device includes a storage module, which is used to: in a case where it is determined that the target data corresponding to the file needs to be written to the storage device, if the size of the target data is greater than a third target value, write the target data to the target storage area N times, wherein the size of the data written to the target storage area each time is less than or equal to the third target value.

[0019] In a possible implementation of the second aspect, when writing the target data to the target storage area, the controller of the storage device updates the value of the management identifier of the target storage area to a second target value, and the second target value indicates that data has been written to the target storage area.

[0020] In a possible implementation of the second aspect, the target storage area includes at least one storage block, and the pre-allocation module is specifically used to: add metadata of one or more data blocks to the metadata corresponding to the file, the metadata of each data block includes the physical address of each data block, and the physical address of each data block maps one or more storage blocks.

[0021] In a possible implementation of the second aspect, the storage device includes a hard disk, and the storage block includes sectors.

[0022] In a third aspect, the present application further provides a computing device, which includes a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the data storage method provided in the first aspect or any possible implementation of the first aspect.

[0023] In a fourth aspect, the present application further provides a computer-readable storage medium, which stores instructions, and when the computer-readable storage medium is run on a computer, enables the computer to execute the data storage method provided by the first aspect or any possible implementation of the first aspect.

[0024] In a fifth aspect, the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the data storage method provided by the first aspect or any possible implementation of the first aspect.

[0025] Any of the above-mentioned devices or computing equipment or computer storage media or computer program products are used to execute the methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding schemes in the corresponding methods provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1a It is a schematic diagram of a scenario of data storage in a computing device provided in an embodiment of the present application;

[0027] Figure 1b This is a schematic diagram of a scenario in which a database system stores data provided by an embodiment of the present application;

[0028] Figure 2a is a schematic diagram of a file pre-allocation solution provided in an embodiment of the present application;

[0029] Figure 2b is a schematic diagram of another file pre-allocation solution provided in an embodiment of the present application;

[0030] Figure 3a is a flow chart of a data storage method provided by an embodiment of the present application;

[0031] Figure 3b It is a schematic diagram of file pre-allocation in a data storage method provided in an embodiment of the present application;

[0032] Figure 4a This is a schematic diagram of multi-threaded parallel writing of data in a database system provided by an embodiment of the present application;

[0033] Figure 4b is a flow chart of another data storage method provided by an embodiment of the present application;

[0034] Figure 5 is a structural schematic diagram of a data storage device provided in an embodiment of the present application;

[0035] Figure 6 It is a structural diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0037] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0038] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refers to two or more systems, and multiple screen terminals refers to two or more screen terminals.

[0039] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0040] Figure 1a Schematic diagram of a data storage scenario in a computing device provided by an embodiment of the present application. Figure 1a As shown, this scenario includes the application layer, kernel layer and device layer.

[0041] The application layer may include an application buffer (APP buffer) and an IO buffer (IO buffer). The IO buffer may include, for example, a C language standard library IO buffer (clib buffer).

[0042] The kernel layer may include page cache, IO queues, and IO drivers.

[0043] The device layer may include storage devices and storage device caches, wherein the APP buffer, IO buffer, and page cache are located in different storage areas of the memory of the computing device.

[0044] The application implements data storage through control flow and data flow. Specifically, Figure 1aAs shown in the figure, the application can write the data stored in the APP buffer into the IO buffer for caching by calling the fwrite interface. Then, the application can write the data cached in the IO buffer to the device layer by calling the fclose interface, or write the data cached in the IO buffer into the page cache by calling the fflush interface (or flush interface) to ensure that the data is not lost.

[0045] In addition, if Figure 1a As shown, the application can also call the write interface to write the data stored in the APP buffer to the page cache of the kernel layer for caching. Then, the kernel calls the fsync interface to write the data cached in the page cache to the device layer. When the kernel calls the fsync interface to write data, it generates a corresponding IO request and adds it to the IO queue, and schedules the IO request in the IO queue to the IO driver according to the pre-set IO scheduling strategy through the IO scheduling algorithm. The IO driver writes the data to the device layer according to the IO request. When writing data to the device layer, taking the storage device as a hard disk as an example, the data can be written to the disk cache first, and then the data in the disk cache can be written to the disk.

[0046] The following uses the application as a database system as an example to introduce the process of storing data in the database.

[0047] Figure 1b It is a schematic diagram of a scenario in which a database system stores data provided in an embodiment of the present application. Figure 1b The database system may include a table disk thread group, a main thread group, and a redo log thread group.

[0048] Each main thread in the main thread group can be used to execute user requests, cache data in the data storage area of ​​the storage device to the kernel's page cache, or modify data in the data storage area, including insert, update, delete, etc. Each table disk thread in the table disk thread group can be used to write table files in the database system from the kernel's page cache to the storage device; each redo log thread in the redo log thread can be used to record transaction logs of user modification operations on data (including), and write the transaction logs to the log storage area of ​​the storage device.

[0049] When each of the above threads reads and writes data to the storage device, it needs to call the interface provided by the kernel's file system. The file system can include different types in specific applications, such as but not limited to file systems such as ext4 and f2fs that manage data files, and file systems such as xfs that manage log files.

[0050] In order to ensure that the file data is continuous on the storage device, it is usually necessary to pre-allocate storage area for the file before writing the corresponding data of the file to the storage device. For example, before the table disk thread and the redo log thread write the file data to the storage device, they can pre-allocate storage area for the file through the file system.

[0051] To prevent the application from reading dirty data in the pre-allocated storage area, file pre-allocation usually includes performing a zero-erasing operation on the pre-allocated target storage area. The following introduces two solutions for performing a zero-erasing operation on the pre-allocated target storage area.

[0052] In solution 1, the application actually writes 0 to the target storage area pre-allocated by the file system through the write interface and fsync interface of the file system, thereby erasing the target storage area. Specifically, the thread of the application can call the write interface provided by the file system to write all-0 data from the database system buffer to the kernel's page cache, and then the application calls the fsync interface of the file system to write all-0 data from the page cache to the pre-allocated target storage area. Figure 2a As shown in FIG. 1 , during the pre-allocation process, data of 0 is stored in each storage block included in the target storage area; during the data storage process, actual data is written into each storage block included in the target storage area ( Figure 2a The * in the figure indicates the actual data).

[0053] During the pre-allocation process, the fsync interface will call the IO interface to write data to the pre-allocated target storage area. If the amount of data to be written is large, the fsync interface will occupy the IO interface for a long time, resulting in a long pre-allocation process. In the case where multiple threads write data to the same storage device, if the fsync interface occupies the IO interface for a long time when processing the IO request of one thread, the fsync interface cannot be called by others, and the IO interface cannot process the IO requests of other threads. The IO requests of other threads are blocked, causing the computing device to freeze.

[0054] To solve the problem of actually writing 0 to the storage area during the pre-allocation process of solution one, a second solution is provided.

[0055] In solution 2, the application program implements zeroing of the storage area by modifying the metadata of the file through the fallocate interface of the file system. In a specific application, the file system of the computing device records the metadata of each file. The metadata of each file includes the metadata of one or more data blocks corresponding to the file. The pre-allocated target storage area includes at least one storage block in the storage device, and each data block maps one or more storage blocks. The file system can read / write the data of the file from the storage block according to the metadata of each data block corresponding to the file. The metadata of the data block may include the physical address, creation time, modification time, etc.

[0056] In this solution, in order to achieve the purpose of zeroing the storage area by modifying the metadata of the file, an identification field is added to the metadata of each data block corresponding to the file. The identification field is used to indicate whether the storage block corresponding to the data block has been written with data. Figure 2b As shown, zero erasure can be achieved by modifying the identification field of each data block in the metadata of the file. The value of the identification field of the modified data block indicates that the storage block corresponding to the data block has not been written with data. For example, after determining the pre-allocated target storage area, the file system can call the IO interface through the fallocate interface to modify the metadata of the file, so that the value of the identification field of the data block mapped by the pre-allocated target storage area is 0, to indicate that the storage block corresponding to each data block has not been written with data. In this way, during the data reading process, if the identification field of the data block indicates that no data has been written, the storage block corresponding to the data block will not be read, which can prevent the application from reading dirty data in the storage block.

[0057] Compared with Solution 1, Solution 2 does not actually write data to the pre-allocated target storage area, occupies the IO interface for a short time, and takes less time to pre-allocate, which is less likely to cause the computing device to freeze. However, after applying Solution 2 for pre-allocation, when the data is written to the disk, in addition to writing the actual data to the storage area, the file system also needs to modify the identification field in the file's metadata again so that the identification field indicates that the data has been written. In other words, when a thread writes data to the pre-allocated target storage area through the file system, the file system needs to call the IO interface twice through the fsync interface, such as Figure 2b As shown, the identification field included in the metadata of the file in the storage device is modified once, and data is written to the pre-allocated target storage area once. Therefore, when the data is written to the disk, the IO interface is called more times and takes a longer time.

[0058] Based on the problems existing in the above two solutions, an embodiment of the present application provides a data storage method.

[0059] In the data storage method provided in the embodiment of the present application, after pre-allocating a storage area for a file, the management identifier of the pre-allocated target storage area is modified by the controller of the storage device to achieve zeroing of the storage area. The controller of the storage device determines whether data is written to the storage area through the management identifier of the storage area, and the modified management identifier of the storage area indicates that data is not written to the storage area. After determining that data is written to the storage area, the controller of the storage device will modify the management identifier of the storage device without the need for the file system to call the IO interface to modify it. In this way, when the application writes data to the storage area through the file system, the file system only needs to call the IO interface once. This embodiment can not only reduce the number of calls to the IO interface during the pre-allocation process, reduce the time occupied by the pre-allocation process for the IO interface, but also reduce the number of calls to the IO interface during the process of data falling to disk, thereby reducing the time occupied by the IO interface during the process of data falling to disk.

[0060] Fig. 3 is a flow chart of a data storage method provided by an embodiment of the present application. The method can be applied to a computing device and executed by a processor of the computing device to store data of files of various applications in the computing device.

[0061] As shown in Figure 3, the method may include the following S301-S303. Specifically, the embodiment of the present application provides a pre-allocation interface, which can be applied to various file systems in the kernel of the computing device. After the file system runs on the processor of the computing device, the pre-allocation interface executes the various steps in Figure 3. Each step is specifically introduced below.

[0062] In S301, the pre-allocation interface receives a pre-allocation request from an application.

[0063] In this step, after the application and the pre-allocation interface are run on the processor of the computing device, when the thread of the application determines that a storage area needs to be pre-allocated for the file, it can call the pre-allocation interface and send a pre-allocation request to the pre-allocation interface.

[0064] The pre-allocation request may include a file identifier and / or a pre-allocated space size, and is used to request pre-allocation of a storage area for a file corresponding to the file identifier. Figure 1b In the case of the database system shown, the file identifier may include a file identifier of a table file and / or a file identifier of a log file in the database system.

[0065] In S302, the pre-allocation interface pre-allocates a target storage area in the storage device for the file corresponding to the file identifier.

[0066] In this step, the computing device may include one or more storage devices, and the pre-allocation interface determines a target storage area that matches the pre-allocated space size from a storage device of the computing device according to the pre-allocated space size. The pre-allocated space size may also be pre-set in the pre-allocation interface. It should be noted that during the pre-allocation process, the target storage area does not store valid data of any application in the computing device. In other words, the target storage area is an unused area.

[0067] After determining the target storage area, the pre-allocation interface pre-allocates the target storage area to the file corresponding to the file identifier in the pre-allocation request. Specifically, the metadata corresponding to the file is updated according to the location information of the target storage area. In a specific application scenario, the target storage area may include at least one storage block in the storage device. In which, taking the storage device as a hard disk as an example, the storage block may include a sector.

[0068] Updating the metadata corresponding to the file according to the location information of the target storage area may include adding metadata of one or more data blocks to the metadata corresponding to the file, wherein the metadata of each data block includes a physical address of each data block, and the physical address of each data block maps one or more storage blocks in the target storage area.

[0069] In S303, the pre-allocation interface instructs the controller of the storage device to update the value of the management identifier corresponding to the target storage area to a first target value, where the first target value indicates that no data has been written into the target storage area.

[0070] Taking a storage device as a hard disk as an example, the hard disk includes multiple sectors. The controller of the hard disk manages the multiple sectors of the hard disk through the identifier of the sector. The identifier of the sector may include information such as the number, address and flag of the sector. The flag of the sector is used to mark whether each sector has been written with data; for example, the value of the flag of the sector may include a first target value and a second target value, the first target value may indicate that the sector has not been written with data, and the second target value may indicate that the sector has been written with data; wherein the first target value may be 0, and the second target value may be 1.

[0071] In this step, the target storage area may include one or more sectors, and the management identifier of the target storage area may include one or more sector flags. The sectors corresponding to the target storage area may have been used before pre-allocation, and the sectors may store invalid data. For the application currently requesting pre-allocation, the invalid data stored in the sectors is dirty data.

[0072] In this step, in order to prevent the application from reading dirty data, the pre-allocation interface can call the IO interface to send a modification request to the controller of the storage device, instructing the controller of the storage device to set the value of the flag bit of each sector corresponding to the target storage area to 0. The modification request may include the number and / or address of each sector corresponding to the target storage area. After receiving the modification request, the controller of the hard disk sets the flag position of each sector to 0 according to the number and / or address of each sector.

[0073] Among them, only some sectors of each sector corresponding to the target storage area may have data written into them. Therefore, in another implementation, the pre-allocation interface can also obtain the flag bits of each sector through the controller of the hard disk, determine whether each sector has been written into the data according to the flag bits of each sector corresponding to the target storage area, and determine the target sector of each sector to which the data has been written into. Then, a modification request is sent to the controller of the storage device, instructing the controller of the storage device to set the flag bit of the target sector to 0. It can be understood that in this case, the modification request can include the number and / or address corresponding to the target sector.

[0074] In this embodiment, after writing the target data into the target storage area, the controller of the storage device sets the value of the flag bit of the sector in the target storage area where the data is written to 1.

[0075] In the above scheme, on the one hand, when the pre-allocation interface of the file system pre-allocates the storage area for the file of the application, it calls the IO interface to instruct the controller of the storage device to modify the management identifier of the pre-allocated target storage area, so as to realize zeroing of the storage area, which can reduce the time that the pre-allocation process occupies the IO interface. On the other hand, since the management identifier indicating whether each storage block in the storage device has been written with data is maintained by the controller of the storage device, after writing the data of the file to the target storage area, the file system does not need to call the IO interface to instruct the hard disk controller to modify the management identifier of the storage block again, which can reduce the number of times the file system calls the IO interface when the data is written to the disk.

[0076] In a computing device, there may be multiple threads writing data to the same storage device. The multiple threads may include different threads of the same application, or threads of different applications. When a thread writes a large amount of data to a storage device, the fsync interface is occupied for a long time, which will cause other threads' data writing tasks to be blocked, causing the computing device to freeze.

[0077] Figure 4a This is a schematic diagram of a database system experiencing transaction jamming provided by an embodiment of the present application.

[0078] In a database system, when a user requests to perform multiple operations on the database system, the main thread of the database system will execute the multiple operations as a transaction, and record the redo log corresponding to the transaction through the redo log thread, and write the redo log to the storage device in real time (that is, the redo log is written to the disk).

[0079] The process of the redo log thread writing the redo log corresponding to the transaction to the disk is as follows: Figure 4a As shown, the redo log thread first determines whether the capacity of the remaining storage area of ​​the APP buffer meets the size of the redo log corresponding to the transaction; if it meets the capacity, the redo log thread writes the redo log into the APP buffer; then, the redo log thread enters the commit phase, calls the write interface to write the redo log into the cache of the memory, and calls the fsync interface to write the redo log into the storage device. After the fsync interface writes the redo log into the storage device, the redo log thread group will return a message to the user indicating that the transaction is successful.

[0080] Since the fsync interface can only be called by one thread at a time, it cannot be called by two threads at the same time. In other words, the fsync interface will be called by the next thread to write the data to the disk only after the previous thread's IO request is completed. Therefore, the redo log thread may wait during the buffer acquisition and commit stages, which will extend the time it takes for the redo log thread to write the log to the disk, and cause transactions to become stuck.

[0081] Combine the following Figure 4a , taking thread 1-thread 3 calling the fsync interface in sequence as an example, the status of the three transactions are introduced respectively.

[0082] When thread 1 requests to flush the redo log corresponding to transaction 1 to disk, the fsync interface is not called by other threads, such as Figure 4a As shown in the figure, the disk writing process of thread 1 proceeds normally. Thread 1 does not need to wait in the buffer acquisition and commit stages. The success time of transaction 1 is not extended and is in a normal state.

[0083] When thread 2 requests to write the redo log corresponding to transaction 2 to disk, if the amount of data written by thread 1 is large and the fsync interface is being called by thread 1, thread 2 needs to wait for thread 1's IO request to be completed before calling the fsync interface. In other words, the commit time of thread 2 is extended, resulting in an extended success time for transaction 2, which is in an abnormal state.

[0084] When thread 3 requests to write the redo log corresponding to transaction 3 to disk, if the amount of data written by thread 2 is large, the storage area of ​​the buffer is used up, and the fsync interface is being called by thread 2. In this case, thread 3 needs to wait for the storage area of ​​the buffer to be released when obtaining the buffer, and in the commit phase, it needs to wait for the IO request of thread 2 to be executed before calling the fsync interface. In other words, the time for thread 3 to obtain the buffer and commit phase are both extended, resulting in an extended time for transaction 3 to succeed, which is in an abnormal state.

[0085] Based on this, an embodiment of the present application provides a data storage method that can solve the above problems.

[0086] Figure 4b 1 is a flow chart of a data storage method provided by an embodiment of the present application. The method can be applied to a computing device and executed by a processor of the computing device. Figure 4b As shown, the method may include S401-S405. Specifically, another file writing interface provided in an embodiment of the present application can be applied to a file system of a computing device. When the file system runs on a processor of the computing device, the file writing interface executes Figure 4b The steps shown.

[0087] Below Figure 4b Each step shown is described in detail.

[0088] In S401, the write file interface determines target data of a file to be written by an application.

[0089] In this step, the write file interface may have the functions of the write interface and the fsync function. Taking the storage device of the computing device as a hard disk as an example, the thread of the application program may call the write file interface, send an IO request to the write file interface, and write the target data to the storage device. Specifically, the write file interface first writes the target data from the APP buffer to the cache of the memory, and after writing the target data to the cache of the memory, writes the target data from the cache of the memory to the storage area of ​​the storage device.

[0090] The IO request may include information such as the identifier of the file, the offset position of the target data to be written corresponding to the file, and the size of the target data.

[0091] In S402, the write file interface determines whether the size of the target data is greater than a third target value.

[0092] In this embodiment, the write file interface can receive IO requests from multiple threads at the same time and implement the functions of the write interface in parallel, that is, write multiple target data into the cache in parallel. When writing each target data into the cache, the write file interface compares the size of the target data and the third target value. In the case where it is determined that the size of the target data is greater than the third target value, the write file interface executes S403 to write the target data into the cache. In the case where it is determined that the size of the target data is less than or equal to the third target value, the write file interface executes S404 to write the target data into the cache.

[0093] The third target value is pre-set in the write file interface, including the maximum capacity of the IO interface corresponding to the hard disk to write data at one time. The third target value can be determined according to the read and write rate of the IO interface corresponding to the hard disk and / or the maximum IO time set by the file system where the write file interface is located. The write file interface can compare the size of the target data with the third target value to determine whether the size of the target data exceeds the pre-set third target value.

[0094] In S403, the write file interface writes the target data into the cache of the memory in N times.

[0095] In this step, taking the target data size as 16M and the third target value as 128K as an example, the write file interface first reads 128K of data from the offset position offset in the IO request, writes it to the cache, and then updates the offset. This process is repeated until all 16M are written to the cache.

[0096] In S404, the write file interface writes the target data into the cache of the memory once.

[0097] In this step, taking the target data size as 64K and the third target value as 128K as an example, the write file interface can write the target data into the cache of the memory at one time.

[0098] In S405, the file writing interface writes each data into the storage device according to the writing order of each data written into the cache.

[0099] In this embodiment, the write file interface implements the fsync function in series, that is, each data is written into the storage device in sequence according to the writing order of each data written into the cache. Since the size of the data written into the cache each time when the write file interface implements the write function is controlled to be no greater than the third target value, the size of the data written into the storage device each time when the write file interface implements the fsync function is also no greater than the third target value. Therefore, the time occupied by the IO interface corresponding to the storage device each time the write file interface writes data into the storage device is relatively small.

[0100] In this embodiment, the data storage method further includes S406, which is executed by a controller of the storage device.

[0101] In S406, the controller of the storage device updates the management identifier corresponding to the storage area where the data is written in the storage device to the second target value. Figure 3b Taking the writing of data into the target storage area of ​​the hard disk as an example, in this step, after writing the target data into the target storage area, the controller of the hard disk sets the value of the flag bit of the sector where the data is written into the target storage area to 1.

[0102] In the above method, when the write file interface writes data to the storage device, if the data to be written is greater than a preset third target value, the data is written to the cache of the memory and the storage device in multiple times, which can reduce the time that the write file interface occupies the IO interface corresponding to the storage device during the data storage process. The write file interface occupies the IO interface for a short time each time. When the write file interface does not occupy the IO interface, other threads can call the write file interface to write data to the storage device, thereby avoiding the IO requests of other threads from being blocked and causing jams. In other words, the write file interface can execute the write tasks of multiple threads in turn, and store part of the data of multiple threads in batches in turn, which can improve the write capability of a single thread of the application. When Figure 4b The method is applied to Figure 4a In the above scenario, the ability to write redo logs corresponding to the transactions of thread 2 and thread 3 to the disk or hard disk can be improved to avoid transaction jams of thread 2 and thread 3.

[0103] In Application Figure 3b and Figure 4b In the computing device of the data storage method shown, when data is written to the disk, only the actual file data needs to be written to the disk or hard disk, and the file metadata does not need to be modified, which can further improve the writing capability of the application in the computing device.

[0104] based on Figure 3b and Figure 4b The data storage method shown, an embodiment of the present application provides a data storage device, which can be used in a computing device.

[0105] Figure 5 1 is a schematic diagram of a data storage device 500 provided in an embodiment of the present application. The data storage device 500 may include a receiving module 501, a pre-allocation module 502 and a storage module 503.

[0106] The receiving module 501 is used to receive a pre-allocation request from an application program, wherein the pre-allocation request includes a file identifier.

[0107] Among them, the pre-allocation module 502 is used to pre-allocate a target storage area in a storage device for the file corresponding to the file identifier, and to instruct the controller of the storage device to update the value of the management identifier corresponding to the target storage area to a first target value, wherein the first target value indicates that no data has been written to the target storage area.

[0108] Among them, the storage module 503 is used to, when it is determined that the target data corresponding to the file needs to be written to the storage device, if the size of the target data is greater than the third target value, write the target data to the target storage area N times, wherein the size of the data written to the target storage area each time is less than or equal to the third target value.

[0109] It should be noted that Figure 5 The data storage device 500 provided in the embodiment shown in the figure only uses the division of the above-mentioned functional modules as an example to illustrate when executing the data storage method. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Figure 3b or Figure 4b The data storage embodiments shown belong to the same concept, and their specific implementation processes are detailed in the method embodiments, which will not be repeated here.

[0110] Figure 6 It is a schematic diagram of the hardware structure of a computing device 600 provided in an embodiment of the present application.

[0111] Figure 6 The computing device 600 shown may include a server, a laptop, a smart phone, etc. Figure 6 The computing device 600 includes a processor 601, a memory 602, a communication interface 603, and a bus 604. The processor 601, the memory 602, and the communication interface 603 are connected to each other via the bus 604. The processor 601, the memory 602, and the communication interface 603 may also be connected in other connection modes besides the bus 604.

[0112] Among them, the memory 602 can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, optical storage, hard disk, etc.

[0113] The processor 601 may be a general-purpose processor, which may be a processor that performs specific steps and / or operations by reading and executing the contents stored in a memory (e.g., memory 602). For example, the general-purpose processor may be a central processing unit (CPU). The processor 601 may include at least one circuit to execute Figure 3a or Figure 4b The illustrated embodiments provide all or part of the steps for data storage.

[0114] The communication interface 603 includes an input / output (I / O) interface, a physical interface, and a logical interface, etc., which are used to interconnect devices within the computing device 600, and an interface for interconnecting the computing device 600 with other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber optic interface, an ATM interface, etc.

[0115] The bus 604 may be any type of communication bus for interconnecting the processor 601 , the memory 602 , and the communication interface 603 , such as a system bus.

[0116] The above devices may be arranged on independent chips, or at least partially or completely on the same chip. Whether to arrange each device independently on different chips or to integrate them on one or more chips often depends on the needs of product design. The embodiments of the present application do not limit the specific implementation form of the above devices.

[0117] Figure 6 The computing device 600 shown is merely exemplary. During implementation, the computing device 600 may further include other components, which are not listed one by one herein.

[0118] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0119] It is understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. It should be understood that in the embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0120] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application should be included in the scope of protection of the present application.

Claims

1. A data storage method, characterized in that: The method comprises: receiving a pre-allocation request from an application, wherein the pre-allocation request includes a file identifier; Pre-allocating a target storage area in a storage device for the file corresponding to the file identifier; The controller of the storage device is instructed to update the value of the management identifier corresponding to the target storage area to a first target value, where the first target value indicates that no data has been written into the target storage area.

2. The method according to claim 1, characterized in that The method further comprises: In the case where it is determined that the target data corresponding to the file needs to be written to the storage device, if the size of the target data is greater than the third target value, the target data is written to the target storage area N times, wherein the size of the data written to the target storage area each time is less than or equal to the third target value.

3. The method according to claim 2, characterized in that The method further comprises: In the case of writing the target data into the target storage area, the controller of the storage device updates the value of the management identifier of the target storage area to a second target value, where the second target value indicates that the data has been written into the target storage area.

4. The method according to any one of claims 1 to 3, characterized in that: The target storage area includes at least one storage block, and the pre-allocation of the target storage area in the storage device for the file corresponding to the file identifier includes: Metadata of one or more data blocks is added to the metadata corresponding to the file, the metadata of each data block includes a physical address of each data block, and the physical address of each data block maps one or more storage blocks.

5. The method according to claim 4, characterized in that The storage device includes a hard disk, and the storage block includes sectors.

6. A data storage device, characterized in that: The data storage device comprises: A receiving module, used for receiving a pre-allocation request from an application, wherein the pre-allocation request includes a file identifier; A pre-allocation module is used to pre-allocate a target storage area in a storage device for a file corresponding to the file identifier, and to instruct a controller of the storage device to update the value of a management identifier corresponding to the target storage area to a first target value, wherein the first target value indicates that no data has been written to the target storage area.

7. The data storage device according to claim 6, characterized in that: The data storage device comprises a storage module, and the storage module is used for: In the case where it is determined that the target data corresponding to the file needs to be written to the storage device, if the size of the target data is greater than the third target value, the target data is written to the target storage area N times, wherein the size of the data written to the target storage area each time is less than or equal to the third target value.

8. The data storage device according to claim 7, characterized in that: In the case of writing the target data into the target storage area, the controller of the storage device updates the value of the management identifier of the target storage area to a second target value, where the second target value indicates that the data has been written into the target storage area.

9. The data storage device according to any one of claims 6 to 8, characterized in that: The target storage area includes at least one storage block, and the pre-allocation module is specifically used for: Metadata of one or more data blocks is added to the metadata corresponding to the file, the metadata of each data block includes a physical address of each data block, and the physical address of each data block maps one or more storage blocks.

10. The method according to claim 9, characterized in that The storage device includes a hard disk, and the storage block includes sectors.

11. A computing device, characterized in that: The computing device comprises: a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the method according to any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 5.

Images