Data storage method and computing device

Abstract

Provided are a data storage method and a computing device. The method comprises: pre-allocating a target storage region in a storage device for a file on the basis of a pre-allocation request of an application program; and instructing a controller of the storage device to update the value of a management identifier corresponding to the target storage region to a first target value, which indicates that no data is written into the target storage region. In the data storage method, a storage region is returned to zero by means of a controller of a storage device modifying a management identifier of a pre-allocated target storage region, such that the time that an IO interface is occupied during a pre-allocation process can be significantly reduced, thereby avoiding the lagging of a computing device caused by the blocking of other IO requests.

Description

Data storage method and computing device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on July 20, 2023, with application number 202310892581.3 and application name “A Data Storage Method and Computing Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a data storage method and a computing device. Background Art

[0003] On a computing device, when the storage area for an application's file is insufficient, it is necessary to pre-allocate storage area on the storage device to the file, ensuring that the corresponding data on the storage device is continuous. Furthermore, to prevent the application from reading dirty data in the pre-allocated target storage area, the pre-allocated target storage area must be erased before writing data to it.

[0004] In the related art, erasing a storage area is achieved by writing 0 to a pre-allocated target storage area. In other words, erasing a storage area requires calling an input / output (IO) interface to write actual data to the storage area, and the written data are all 0. This erasing method takes a long time, causing the IO interface to be occupied for a long time. In the case of pre-allocating storage areas for multiple files, the IO interface being occupied for a long time will cause other IO requests to be blocked, which in turn causes the computing device to freeze, affecting the user experience.

[0005] Summary of the Invention

[0006] The present application provides a data storage method and a computing device, which can significantly reduce the time that an IO interface is occupied during the pre-allocation process by modifying the management identifier of a pre-allocated target storage area to erase the storage area.

[0007] In a first aspect, the present application provides a data storage method. The method can be applied to a computing device, comprising: receiving a pre-allocation request from an application, the pre-allocation request including a file identifier; pre-allocating a target storage area in a storage device for a file corresponding to the file identifier; and instructing a controller of the storage device to update the value of a management identifier corresponding to the target storage area to a first target value, the first target value indicating that no data has been written to the target storage area.

[0008] In the above solution, during the pre-allocation process, the storage device controller modifies the management identifier corresponding to the target storage area and erases the target storage area to zero, which can reduce the time the storage device IO interface is occupied during the pre-allocation process.

[0009] In a possible implementation of the first aspect, the method further includes: in a case where it is determined that the target data corresponding to the file needs to be written to the storage device, if the size of the target data is greater than a third target value, writing the target data to the target storage area N times, wherein the size of the data written to the target storage area each time is less than or equal to the third target value.

[0010] In the above scheme, when the target data exceeds the third target value, the target data is written to the storage device in multiple times. By controlling the amount of data written to the storage device each time, the time the IO interface of the storage device is occupied during a single write can be reduced, thereby avoiding the situation where the IO interface is occupied for a long time due to a large amount of written data, causing other IO requests to be blocked and causing the computing device to freeze.

[0011] In a possible implementation of the first aspect, the method further includes: in the case of writing the target data into the target storage area, the controller of the storage device updates the value of the management identifier of the target storage area to a second target value, and the second target value indicates that data has been written into the target storage area.

[0012] In the above solution, after the data is written into the storage device, the controller modifies the management identifier of the target storage area, which can reduce the number of calls to the IO interface of the storage device during the data writing process.

[0013] In a possible implementation of the first aspect, the target storage area includes at least one storage block, and pre-allocating the target storage area in the storage device for the file corresponding to the file identifier includes: adding metadata of one or more data blocks to the metadata corresponding to the file, the metadata of each data block including the physical address of each data block, and the physical address of each data block maps one or more storage blocks.

[0014] In the above solution, the target storage area is associated with the file through metadata, which makes it easy to read / write the file data to the target storage area.

[0015] In a possible implementation of the first aspect, the storage device includes a hard disk, and the storage block includes a sector.

[0016] In the above solution, the pre-allocation process described in the first aspect can be implemented on the hard disk, reducing the time occupied by the pre-allocation process on the hard disk's IO interface and reducing the number of calls to the hard disk's IO interface when writing data to the hard disk.

[0017] In a second aspect, the present application further provides a data storage device, which can be applied to a computing device and includes: a receiving module and a pre-allocation module.

[0018] The receiving module is used to receive a pre-allocation request from an application program, wherein the pre-allocation request includes a file identifier.

[0019] Among them, the pre-allocation module is used to pre-allocate a target storage area in the storage device for the file corresponding to the file identifier, and instruct the controller of the storage device to update the value of the management identifier corresponding to the target storage area to a first target value, wherein the first target value indicates that no data has been written to the target storage area.

[0020] In a possible implementation of the second aspect, the data storage device includes a storage module, which is used to: in a case where it is determined that the target data corresponding to the file needs to be written to the storage device, if the size of the target data is greater than a third target value, write the target data to the target storage area N times, wherein the size of the data written to the target storage area each time is less than or equal to the third target value.

[0021] In a possible implementation of the second aspect, when the target data is written to the target storage area, the controller of the storage device updates the value of the management identifier of the target storage area to a second target value, and the second target value indicates that data has been written to the target storage area.

[0022] In a possible implementation of the second aspect, the target storage area includes at least one storage block, and the pre-allocation module is specifically used to: add metadata of one or more data blocks to the metadata corresponding to the file, the metadata of each data block includes the physical address of each data block, and the physical address of each data block maps one or more storage blocks.

[0023] In a possible implementation of the second aspect, the storage device includes a hard disk, and the storage block includes a sector.

[0024] In a third aspect, the present application further provides a computing device comprising: a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the data storage method provided in the first aspect or any possible implementation of the first aspect.

[0025] In a fourth aspect, the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the data storage method provided by the above-mentioned first aspect or any possible implementation of the first aspect.

[0026] In a fifth aspect, the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the data storage method provided by the first aspect or any possible implementation of the first aspect.

[0027] Any of the above-mentioned devices, computing equipment, computer storage media, or computer program products is used to execute the method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding schemes in the corresponding methods provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] FIG1a is a schematic diagram of a data storage scenario in a computing device provided by an embodiment of the present application;

[0029] FIG1b is a schematic diagram of a scenario in which a database system stores data according to an embodiment of the present application;

[0030] FIG2a is a schematic diagram of a file pre-allocation solution provided in an embodiment of the present application;

[0031] FIG2 b is a schematic diagram of another file pre-allocation solution provided in an embodiment of the present application;

[0032] FIG3a is a flow chart of a data storage method provided in an embodiment of the present application;

[0033] FIG3 b is a schematic diagram of file pre-allocation in a data storage method provided in an embodiment of the present application;

[0034] FIG4a is a schematic diagram of multi-threaded parallel data writing in a database system provided by an embodiment of the present application;

[0035] FIG4 b is a flow chart of another data storage method provided in an embodiment of the present application;

[0036] FIG5 is a schematic structural diagram of a data storage device provided in an embodiment of the present application;

[0037] FIG6 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0039] In the description of the embodiments of the present application, words such as "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of the present application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0040] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "plurality" means two or more. For example, "multiple systems" refers to two or more systems, and "multiple screen terminals" refers to two or more screen terminals.

[0041] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly identifying the technical features being referred to. Thus, features specified as "first" or "second" may explicitly or implicitly include one or more of such features. The terms "include," "comprising," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0042] Figure 1a is a schematic diagram of a data storage scenario in a computing device provided by an embodiment of the present application. As shown in Figure 1a, the scenario includes an application layer, a kernel layer, and a device layer.

[0043] The application layer may include an application buffer (APP buffer) and an IO buffer (IO buffer). The IO buffer may include, for example, a C language standard library IO buffer (clib buffer).

[0044] The kernel layer may include page cache, IO queues, and IO drivers.

[0045] The device layer can include storage devices and storage device caches. Among them, the APP buffer, IO buffer, and page cache are located in different storage areas of the computing device's memory.

[0046] Applications implement data storage through control and data flows. Specifically, as shown in Figure 1a, an application can call the fwrite interface to write data stored in the app buffer to the IO buffer for caching. The application can then call the fclose interface to write the data cached in the IO buffer to the device layer, or call the fflush interface (or flush interface) to write the data cached in the IO buffer to the page cache for caching, ensuring data is not lost.

[0047] In addition, as shown in Figure 1a, the application can also call the write interface to write the data stored in the APP buffer to the kernel layer's page cache for caching. The kernel then calls the fsync interface to write the data cached in the page cache to the device layer. When the kernel calls the fsync interface to write data, it generates a corresponding IO request and adds it to the IO queue. The kernel then uses the IO scheduling algorithm to dispatch the IO request in the IO queue to the IO driver according to the pre-set IO scheduling policy. The IO driver then writes the data to the device layer according to the IO request. When writing data to the device layer, taking the storage device as a hard disk as an example, the data can be written to the disk cache first, and then the data in the disk cache can be written to the disk.

[0048] The following takes the application as a database system as an example to introduce the process of storing data in the database.

[0049] Figure 1b is a schematic diagram of a scenario in which a database system stores data according to an embodiment of the present application. Figure 1b includes a table flushing thread group, a main thread group, and a redo log thread group in the database system.

[0050] Each main thread in the main thread group can be used to execute user requests, cache data in the storage device's data storage area to the kernel's page cache, or modify data in the data storage area, including inserts, updates, and deletes. Each table flushing thread in the table flushing thread group can be used to write table files in the database system from the kernel's page cache to the storage device; each redo log thread in the redo log thread group can be used to record transaction logs of (including) user data modification operations and write the transaction logs to the log storage area of ​​the storage device.

[0051] Each of these threads must invoke the kernel's file system interface when reading and writing data to the storage device. File systems can come in many different types, including but not limited to ext4 and f2fs for managing data files, and xfs for managing log files.

[0052] To ensure file data continuity on storage devices, it's often necessary to pre-allocate storage space for the file before writing the corresponding data to the storage device. For example, the table flushing thread and redo log thread can pre-allocate storage space for the file through the file system before writing the file data to the storage device.

[0053] To prevent applications from reading dirty data in the preallocated storage area, file preallocation typically includes erasing the preallocated target storage area. The following describes two methods for erasing the preallocated target storage area.

[0054] In solution one, the application actually writes zeros to the target storage area pre-allocated by the file system through the write and fsync interfaces of the file system, thereby erasing the target storage area. Specifically, the application thread can call the write interface provided by the file system to write all-zero data from the database system's buffer to the kernel's page cache. The application then calls the file system's fsync interface to write all-zero data from the page cache to the pre-allocated target storage area. As shown in Figure 2a, during the pre-allocation process, data of zero is stored in each storage block included in the target storage area; during the data write process, the actual data is written to each storage block included in the target storage area (the * in Figure 2a represents the actual data).

[0055] During the pre-allocation process, the fsync interface calls the IO interface to write data to the pre-allocated target storage area. If the amount of data to be written is large, the fsync interface will occupy the IO interface for a long time, causing the pre-allocation process to take a long time. When multiple threads write data to the same storage device, if the fsync interface occupies the IO interface for a long time when processing the IO request of one thread, the fsync interface cannot be called by other threads, and the IO interface cannot process the IO requests of other threads. The IO requests of other threads are blocked, causing the computing device to freeze.

[0056] To solve the problem of actually writing 0 to the storage area during the pre-allocation process of solution 1, a second solution is provided.

[0057] In solution two, the application uses the file system's fallocate interface to modify the file's metadata to achieve zero-erasing of the storage area. In specific applications, the computing device's file system records metadata for each file. Each file's metadata includes metadata for one or more data blocks corresponding to the file. The pre-allocated target storage area includes at least one storage block in the storage device, with each data block mapping to one or more storage blocks. The file system can read and write the file's data from the storage blocks based on the metadata for each data block. Data block metadata may include physical addresses, creation times, modification times, and more.

[0058] In this solution, in order to achieve the purpose of erasing the storage area by modifying the metadata of the file, an identification field is added to the metadata of each data block corresponding to the file. The identification field is used to indicate whether the storage block corresponding to the data block has data written into it. In this way, after pre-allocating the storage area for the file, as shown in Figure 2b, erasing can be achieved by modifying the identification field of each data block in the metadata of the file. The value of the identification field of the modified data block indicates that the storage block corresponding to the data block has not been written into. For example, after determining the pre-allocated target storage area, the file system can call the IO interface through the fallocate interface to modify the metadata of the file so that the value of the identification field of the data block mapped to the pre-allocated target storage area is 0, indicating that the storage block corresponding to each data block has not been written into it. In this way, during the data reading process, if the identification field of the data block indicates that no data has been written into it, the storage block corresponding to the data block will not be read, which can prevent the application from reading dirty data in the storage block.

[0059] Compared to Solution 1, Solution 2 does not actually write data to the pre-allocated target storage area, occupies the IO interface for a short time, and the pre-allocation takes a short time, which is less likely to cause the computing device to freeze. However, after applying Solution 2 for pre-allocation, when the data is written to the disk, in addition to writing the actual data to the storage area, the file system also needs to modify the identification field in the file's metadata again so that the identification field indicates that the data has been written. In other words, when a thread writes data to the pre-allocated target storage area through the file system, the file system needs to call the IO interface twice through the fsync interface, as shown in Figure 2b, once to modify the identification field included in the metadata of the file in the storage device, and once to write data to the pre-allocated target storage area. Therefore, when the data is written to the disk, the IO interface is called more times and takes a longer time.

[0060] Based on the problems existing in the above two solutions, an embodiment of the present application provides a data storage method.

[0061] In the data storage method provided in the embodiment of the present application, after pre-allocating a storage area for a file, the management identifier of the pre-allocated target storage area is modified by the controller of the storage device to achieve zero erasure of the storage area. The controller of the storage device determines whether data has been written to the storage area through the management identifier of the storage area, and the modified management identifier of the storage area indicates that data has not been written to the storage area. After determining that data is written to the storage area, the controller of the storage device will modify the management identifier of the storage device without the need for the file system to call the IO interface to modify it. In this way, when the application writes data to the storage area through the file system, the file system only needs to call the IO interface once. This embodiment can not only reduce the number of calls to the IO interface during the pre-allocation process and reduce the time occupied by the IO interface during the pre-allocation process, but also reduce the number of calls to the IO interface during the data placement process, thereby reducing the time occupied by the IO interface during the data placement process.

[0062] Figure 3 is a flow chart of a data storage method provided by an embodiment of the present application. The method can be applied to a computing device and executed by a processor of the computing device to store data of files of various applications in the computing device.

[0063] As shown in Figure 3, the method may include the following steps S301-S303. Specifically, an embodiment of the present application provides a pre-allocation interface that can be applied to various file systems in the kernel of a computing device. After the file system runs on the processor of the computing device, the pre-allocation interface executes the steps in Figure 3. Each step is described in detail below.

[0064] In S301 , a pre-allocation interface receives a pre-allocation request from an application.

[0065] In this step, after the application and the pre-allocation interface are run on the processor of the computing device, the thread of the application can call the pre-allocation interface and send a pre-allocation request to the pre-allocation interface when determining that a storage area needs to be pre-allocated for the file.

[0066] The pre-allocation request may include a file identifier and / or a pre-allocated space size, and is used to request pre-allocation of a storage area for the file corresponding to the file identifier. In the case where the application is the database system shown in FIG1 b , the file identifier may include a file identifier of a table file and / or a file identifier of a log file in the database system.

[0067] In S302, the pre-allocation interface pre-allocates a target storage area in the storage device for the file corresponding to the file identifier.

[0068] In this step, the computing device may include one or more storage devices. The pre-allocation interface determines, based on the pre-allocated space size, a target storage area from a storage device of the computing device that matches the pre-allocated space size. The pre-allocated space size may also be pre-set in the pre-allocation interface. It should be noted that during the pre-allocation process, the target storage area does not store valid data for any application in the computing device. In other words, the target storage area is an unused area.

[0069] After determining the target storage area, the pre-allocation interface pre-allocates the target storage area to the file corresponding to the file identifier in the pre-allocation request. Specifically, the metadata corresponding to the file is updated based on the location information of the target storage area. In specific application scenarios, the target storage area may include at least one storage block in a storage device. For example, in the case of a hard disk, a storage block may include a sector.

[0070] Updating metadata corresponding to the file based on the location information of the target storage area may include adding metadata of one or more data blocks to the metadata corresponding to the file, wherein the metadata of each data block includes a physical address of each data block, and the physical address of each data block maps to one or more storage blocks in the target storage area.

[0071] In S303, the pre-allocation interface instructs the controller of the storage device to update the value of the management identifier corresponding to the target storage area to a first target value, where the first target value indicates that no data has been written into the target storage area.

[0072] Taking a hard disk as an example, the storage device includes multiple sectors. The hard disk controller manages the multiple sectors of the hard disk using sector identifiers. The sector identifier may include information such as the sector number, address, and flag bit. The sector flag bit is used to indicate whether each sector has data written to it. For example, the value of the sector flag bit may include a first target value and a second target value. The first target value may indicate that the sector has not been written to, while the second target value indicates that the sector has been written to. The first target value may be 0, and the second target value may be 1.

[0073] In this step, the target storage area may include one or more sectors, and the management identifier of the target storage area may include one or more sector flags. The sectors corresponding to the target storage area may have been used before pre-allocation, and the sectors may store invalid data. For the application currently requesting pre-allocation, the invalid data stored in the sectors is considered dirty data.

[0074] In this step, to prevent the application from reading dirty data, the pre-allocation interface can call the IO interface to send a modification request to the storage device controller, instructing the storage device controller to set the flag bits of each sector corresponding to the target storage area to 0. The modification request can include the number and / or address of each sector corresponding to the target storage area. After receiving the modification request, the hard disk controller sets the flag bits of each sector to 0 based on the number and / or address of each sector.

[0075] Among these, only some sectors corresponding to the target storage area may have data written to them. Therefore, in another implementation, the pre-allocation interface can also obtain the flag bits of each sector from the hard disk controller, determine whether each sector has data written to it based on the flag bits of each sector corresponding to the target storage area, and determine the target sector among the sectors to which data has been written. Then, a modification request is sent to the storage device controller, instructing the storage device controller to set the flag bit of the target sector to 0. It will be appreciated that in this case, the modification request can include the number and / or address corresponding to the target sector.

[0076] In this embodiment, after writing the target data into the target storage area, the controller of the storage device sets the value of the flag bit of the sector in the target storage area where the data is written to 1.

[0077] In the above scheme, on the one hand, when the file system's pre-allocation interface pre-allocates storage area for an application's file, it calls the IO interface to instruct the storage device's controller to modify the management identifier of the pre-allocated target storage area, thereby erasing the storage area and reducing the time the IO interface is occupied by the pre-allocation process. On the other hand, because the management identifier indicating whether each storage block in the storage device has data written is maintained by the storage device's controller, after writing the file's data to the target storage area, the file system does not need to call the IO interface to instruct the hard disk controller to modify the storage block's management identifier again, thus reducing the number of IO interface calls required by the file system when data is written to the disk.

[0078] In a computing device, multiple threads may write data to the same storage device. These threads can include different threads of the same application or threads of different applications. When a thread writes a large amount of data to a storage device, the fsync interface is occupied for a long time, blocking the data write tasks of other threads and causing the computing device to freeze.

[0079] FIG4 a is a schematic diagram of a database system experiencing transaction jamming, provided by an embodiment of the present application.

[0080] In a database system, when a user requests to perform multiple operations on the database system, the main thread of the database system will execute the multiple operations as a transaction, and record the redo log corresponding to the transaction through the redo log thread, and write the redo log to the storage device in real time (that is, the redo log is written to the disk).

[0081] Figure 4a shows the process by which the redo log thread writes the redo log corresponding to a transaction to disk. The redo log thread first determines whether the remaining storage area in the APP buffer has the capacity required for the redo log corresponding to the transaction. If so, the redo log thread writes the redo log to the APP buffer. The redo log thread then enters the commit phase, calling the write interface to write the redo log to the memory cache and the fsync interface to write the redo log to the storage device. Only after the fsync interface writes the redo log to the storage device does the redo log thread group return a message to the user indicating the success of the transaction.

[0082] Because the fsync interface can only be called by one thread at a time, not by two threads simultaneously, the fsync interface is called by the next thread to flush its data to disk only after the previous thread completes its I / O request. Consequently, the redo log thread may experience wait times during buffer acquisition and commit operations, prolonging the redo log flushing process and causing transaction stalls.

[0083] The following describes the status of three transactions respectively, with reference to FIG4a, taking thread 1 to thread 3 calling the fsync interface in sequence as an example.

[0084] When thread 1 requests to flush the redo log corresponding to transaction 1 to disk, the fsync interface is not called by other threads. As shown in Figure 4a, the flushing process of thread 1 proceeds normally. Thread 1 does not need to wait during the buffer acquisition and commit phases. The success time of transaction 1 is not extended and it remains in a normal state.

[0085] When thread 2 requests to flush the redo log corresponding to transaction 2 to disk, if thread 1 is writing a large amount of data and the fsync interface is being called by thread 1, thread 2 must wait for thread 1's I / O request to complete before calling the fsync interface during the commit phase. This prolongs thread 2's commit phase, delaying the success of transaction 2 and causing an abnormal state.

[0086] When thread 3 requests to flush the redo log corresponding to transaction 3 to disk, if thread 2 has written a large amount of data, the buffer storage area will be exhausted and the fsync interface is being called by thread 2. In this situation, thread 3 must wait for the buffer storage area to be released before obtaining the buffer. Furthermore, during the commit phase, thread 3 must wait for thread 2's I / O request to complete before calling the fsync interface. This prolongs both the buffer acquisition and commit phases for thread 3, delaying the success of transaction 3 and causing an abnormal state.

[0087] Based on this, an embodiment of the present application provides a data storage method that can solve the above problems.

[0088] Figure 4b is a flow chart of a data storage method provided by an embodiment of the present application. This method can be applied to a computing device and executed by a processor of the computing device. As shown in Figure 4b, the method may include steps S401-S405. Specifically, another file write interface provided by an embodiment of the present application can be applied to a file system of a computing device. When the file system is running on the processor of the computing device, the file write interface executes the steps shown in Figure 4b.

[0089] The following is a detailed introduction to each step shown in FIG4b.

[0090] In S401 , the write file interface determines target data of a file to be written by the application.

[0091] In this step, the file write interface can have both the write interface and fsync functionality. For example, if the computing device's storage device is a hard disk, the application's thread can call the file write interface and send an IO request to the interface to write the target data to the storage device. Specifically, the file write interface first writes the target data from the app buffer to the memory cache. After writing the target data to the memory cache, the target data is then written from the memory cache to the storage area of ​​the storage device.

[0092] The IO request may include information such as the file identifier, the offset position of the target data to be written corresponding to the file, and the size of the target data.

[0093] In S402 , the write file interface determines whether the size of the target data is greater than a third target value.

[0094] In this embodiment, the write file interface can simultaneously receive I / O requests from multiple threads and implement the functionality of the write interface in parallel, namely, writing multiple target data into the cache in parallel. When writing each target data into the cache, the write file interface compares the size of the target data with a third target value. If the size of the target data is determined to be greater than the third target value, the write file interface executes S403 to write the target data into the cache. If the size of the target data is determined to be less than or equal to the third target value, the write file interface executes S404 to write the target data into the cache.

[0095] The third target value is pre-set in the file write interface and includes the maximum capacity of data that can be written at one time by the IO interface corresponding to the hard disk. The third target value can be determined based on the read and write rates of the IO interface corresponding to the hard disk and / or the maximum IO time set by the file system where the file write interface resides. The file write interface can compare the size of the target data with the third target value to determine whether the size of the target data exceeds the pre-set third target value.

[0096] In S403, the file writing interface writes the target data into the cache of the memory in N times.

[0097] In this step, assuming the target data size is 16MB and the third target value is 128KB, the write file interface first reads 128KB of data starting from the offset in the IO request, writes it to the cache, and then updates the offset. This process repeats until all 16MB of data is written to the cache.

[0098] In S404, the write file interface writes the target data into the cache of the memory once.

[0099] In this step, taking the target data size as 64K and the third target value as 128K as an example, the write file interface can write the target data into the cache of the memory at one time.

[0100] In S405 , the file writing interface writes each data into the storage device according to the writing order of each data into the cache.

[0101] In this embodiment, the write file interface implements the fsync function serially, that is, each data item is written to the storage device sequentially according to the order in which the data items are written to the cache. Because the size of each data item written to the cache by the write file interface is controlled to be no larger than a third target value when implementing the write function, the size of each data item written to the storage device by the write file interface is also no larger than the third target value when implementing the fsync function. Therefore, each time the write file interface writes data to the storage device, the time occupied by the storage device's corresponding IO interface is relatively short.

[0102] In this embodiment, the data storage method further includes S406, which is executed by a controller of the storage device.

[0103] In S406, the controller of the storage device updates the management identifier corresponding to the storage area in the storage device where the data is written to the second target value. Taking writing data to the target storage area of ​​the hard disk shown in FIG3b as an example, in this step, after writing the target data to the target storage area, the controller of the hard disk sets the flag bit of the sector in the target storage area where the data is written to 1.

[0104] In the above method, when the write file interface writes data to the storage device, if the data to be written is greater than a pre-set third target value, the data is written to the cache of the memory and the storage device in multiple times, which can reduce the time that the write file interface occupies the IO interface corresponding to the storage device during the data storage process. The write file interface occupies the IO interface for a short time each time. When the write file interface does not occupy the IO interface, other threads can call the write file interface to write data to the storage device, thereby avoiding the IO requests of other threads from being blocked and causing jams. In other words, the write file interface can execute the write tasks of multiple threads in turn, and write part of the data of multiple threads in batches in turn, which can improve the write capability of a single thread of the application. When the method described in Figure 4b is applied to the scenario described in Figure 4a, the ability of the redo logs corresponding to the transactions of thread 2 and thread 3 to be written to the disk or hard disk can be improved, avoiding jams in the transactions of thread 2 and thread 3.

[0105] In a computing device that applies the data storage method shown in Figures 3b and 4b, when data is written to the disk, only the actual file data needs to be written to the disk or hard disk, and there is no need to modify the file metadata, which can further improve the writing capability of the application in the computing device.

[0106] Based on the data storage method shown in FIG. 3 b and FIG. 4 b , an embodiment of the present application provides a data storage device, which can be used in a computing device.

[0107] 5 is a schematic diagram of the structure of a data storage device 500 provided in an embodiment of the present application. The data storage device 500 may include a receiving module 501 , a pre-allocation module 502 , and a storage module 503 .

[0108] The receiving module 501 is configured to receive a pre-allocation request from an application program, wherein the pre-allocation request includes a file identifier.

[0109] Among them, the pre-allocation module 502 is used to pre-allocate a target storage area in the storage device for the file corresponding to the file identifier, and instruct the controller of the storage device to update the value of the management identifier corresponding to the target storage area to a first target value, wherein the first target value indicates that no data has been written to the target storage area.

[0110] Among them, the storage module 503 is used to write the target data corresponding to the file into the storage device for N times if it is determined that the target data corresponding to the file needs to be written into the storage device, if the size of the target data is greater than the third target value, wherein the size of the data written to the target storage area each time is less than or equal to the third target value.

[0111] It should be noted that the data storage device 500 provided in the embodiment shown in FIG5 is merely an example of the division of the aforementioned functional modules when executing the data storage method. In actual applications, the aforementioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the computing device provided in the above embodiment and the data storage embodiment shown in FIG3b or FIG4b are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0112] FIG6 is a schematic diagram of the hardware structure of a computing device 600 provided in an embodiment of the present application.

[0113] The computing device 600 shown in FIG6 may include a server, a laptop computer, a smartphone, etc. Referring to FIG6 , the computing device 600 includes a processor 601, a memory 602, a communication interface 603, and a bus 604. The processor 601, the memory 602, and the communication interface 603 are interconnected via the bus 604. The processor 601, the memory 602, and the communication interface 603 may also be connected using other connection methods besides the bus 604.

[0114] Among them, the memory 602 can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, optical storage, hard disk, etc.

[0115] Processor 601 may be a general-purpose processor that performs specific steps and / or operations by reading and executing content stored in a memory (e.g., memory 602). For example, the general-purpose processor may be a central processing unit (CPU). Processor 601 may include at least one circuit to perform all or part of the steps of data storage provided in the embodiment shown in FIG. 3a or FIG. 4b.

[0116] Communication interface 603 includes input / output (I / O) interfaces, physical interfaces, and logical interfaces, which are used to interconnect components within computing device 600, as well as interfaces for interconnecting computing device 600 with other devices (e.g., other computing devices or user equipment). Physical interfaces can include Ethernet interfaces, fiber optic interfaces, ATM interfaces, and the like.

[0117] The bus 604 may be any type of communication bus for interconnecting the processor 601 , the memory 602 , and the communication interface 603 , such as a system bus.

[0118] The above-mentioned devices can be provided on separate chips, or at least partially or entirely on the same chip. Whether to provide each device independently on different chips or to integrate them on one or more chips often depends on the product design requirements. The embodiments of this application do not limit the specific implementation of the above-mentioned devices.

[0119] The computing device 600 shown in FIG6 is merely exemplary. During implementation, the computing device 600 may further include other components, which are not listed here one by one.

[0120] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0121] It is understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not intended to limit the scope of the embodiments of the present application. It should be understood that in the embodiments of the present application, the order of the sequence numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0122] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of this application in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of this application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of this application should be included in the scope of protection of this application.

Claims

1. A data storage method, characterized in that: The method comprises: receiving a pre-allocation request from an application, wherein the pre-allocation request includes a file identifier; Pre-allocating a target storage area in a storage device for the file corresponding to the file identifier; The controller of the storage device is instructed to update the value of the management identifier corresponding to the target storage area to a first target value, where the first target value indicates that no data has been written into the target storage area.

2. The method according to claim 1, characterized in that The method further comprises: In the case where it is determined that the target data corresponding to the file needs to be written to the storage device, if the size of the target data is greater than the third target value, the target data is written to the target storage area N times, wherein the size of the data written to the target storage area each time is less than or equal to the third target value.

3. The method according to claim 2, characterized in that The method further comprises: In the case of writing the target data into the target storage area, the controller of the storage device updates the value of the management identifier of the target storage area to a second target value, where the second target value indicates that the data has been written into the target storage area.

4. The method according to any one of claims 1 to 3, characterized in that: The target storage area includes at least one storage block, and the pre-allocation of the target storage area in the storage device for the file corresponding to the file identifier includes: Metadata of one or more data blocks is added to the metadata corresponding to the file, the metadata of each data block includes a physical address of each data block, and the physical address of each data block maps one or more storage blocks.

5. The method according to claim 4, characterized in that The storage device includes a hard disk, and the storage block includes sectors.

6. A data storage device, characterized in that: The data storage device comprises: A receiving module, used for receiving a pre-allocation request from an application, wherein the pre-allocation request includes a file identifier; A pre-allocation module is used to pre-allocate a target storage area in a storage device for a file corresponding to the file identifier, and to instruct a controller of the storage device to update the value of a management identifier corresponding to the target storage area to a first target value, wherein the first target value indicates that no data has been written to the target storage area.

7. The data storage device according to claim 6, characterized in that: The data storage device comprises a storage module, and the storage module is used for: In the case where it is determined that the target data corresponding to the file needs to be written to the storage device, if the size of the target data is greater than the third target value, the target data is written to the target storage area N times, wherein the size of the data written to the target storage area each time is less than or equal to the third target value.

8. The data storage device according to claim 7, characterized in that: In the case of writing the target data into the target storage area, the controller of the storage device updates the value of the management identifier of the target storage area to a second target value, where the second target value indicates that the data has been written into the target storage area.

9. The data storage device according to any one of claims 6 to 8, characterized in that: The target storage area includes at least one storage block, and the pre-allocation module is specifically used for: Metadata of one or more data blocks is added to the metadata corresponding to the file, the metadata of each data block includes a physical address of each data block, and the physical address of each data block maps one or more storage blocks.

10. The method according to claim 9, characterized in that The storage device includes a hard disk, and the storage block includes sectors.

11. A computing device, characterized in that: The computing device comprises: a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the method according to any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 5.

13. A computer program product, characterized in that The method comprises instructions, which, when executed on a computer, enable the computer to implement the method according to any one of claims 1 to 5.

Images