Data processing method and device and electronic equipment

Abstract

The embodiment of the invention provides a data processing method and device and electronic equipment. In the embodiment, when a data writing request is received, whether a target cache region corresponding to a target file indicated by the request exists locally or not is checked firstly, if not, the target cache region is created in a local memory, and the size of the target cache region is the same as that of a storage page in an NAND Flash medium; and then, before to-be-written target data is written into the NAND Flash medium, the target data is written into the target cache region based on the size of the available storage space of the target cache region, and on this basis, when the size of the available storage space in the target cache region is a preset value, namely when the target cache region is full, the data in the target cache region is written into the NAND Flash medium. In this way, write amplification caused by the fact that the size of the target data to be written is smaller than that of the storage page can be effectively avoided, and therefore the PE service life of the NAND Flash medium is prolonged.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to a data processing method, apparatus, and electronic device. Background Art

[0002] The Nand flash medium is a non-volatile storage medium, which has a high sequential write speed, a large storage density, and a high PE lifespan, and is widely used in large-capacity data storage. Here, PE is the abbreviation of Program / Erase. The structure of the Nand flash medium includes multiple blocks (which can be denoted as storage blocks), and each block includes multiple pages (which can be denoted as storage pages).

[0003] In the scenario of log data storage, since the minimum write unit in the Nand flash medium is usually a Page, and the minimum erase unit is usually a Block; based on this, if the amount of data to be written indicated by the data write request is less than the size of a Page, it will cause the current data write operation to occupy the write amount of the entire Page, resulting in write amplification; if all available Blocks in the Nand flash medium are occupied, a rotate operation will be triggered, that is, deleting old data files, and deletion usually involves data migration, such as migrating the valid data in the Block to be deleted to other Blocks, and then erasing the data in the Block to be deleted, which will cause the actual amount of data written to be greater than the amount of data to be written indicated by the above data write request, resulting in write amplification; and write amplification will reduce the PE lifespan of the Nand flash medium; where write amplification can refer to the phenomenon that the amount of data actually written to the storage medium is greater than the amount of data expected to be written by the data write request. Therefore, how to effectively reduce write amplification to improve the PE lifespan of the Nand flash medium is a technical problem that needs to be solved urgently at present. Summary of the Invention

[0004] In view of this, this application provides a data processing method, apparatus, and electronic device to reduce write amplification and improve the PE lifespan of the Nand flash medium.

[0005] An embodiment of this application further provides a data processing method, which is applied to an electronic device, and the method includes:

[0006] Receiving a data write request; the data write request indicates the identifier of the target file;

[0007] Check whether a target buffer area corresponding to the target file with a specified size exists locally based on the identifier of the target file. If not, create a target buffer area corresponding to the target file in the local memory; the specified size is the size of a storage page in the NAND Flash medium of the electronic device;

[0008] If the size of the available storage space in the target buffer area corresponding to the target file is less than the size of the target data to be written indicated by the data write request, before writing the target data to the NAND Flash medium, truncate the target data to obtain first-part data and second-part data, write the first-part data to the available storage space in the target buffer area, the size of the first-part data matching the size of the available storage space in the target buffer area, then write the data in the target buffer area to the NAND Flash medium, and clear the data in the target buffer area, and write the second-part data to the available storage space in the target buffer area based on the size of the available storage space in the target buffer area; and when the size of the available storage space in the target buffer area is a preset value, write the data in the target buffer area to the NAND Flash medium, and clear the data in the target buffer area;

[0009] If the size of the available storage space in the target buffer area corresponding to the target file is greater than or equal to the size of the target data, before writing the target data to the NAND Flash medium, write the target data to the available storage space in the target buffer area, and when the size of the available storage space in the target buffer area is a preset value, write the data in the target buffer area to the NAND Flash medium, and clear the data in the target buffer area.

[0010] An embodiment of this application also provides a data processing device, which is applied to an electronic device. The device includes:

[0011] A receiving module, configured to receive a data write request; the data write request indicates the identifier of the target file;

[0012] An inspection module, configured to check whether a target buffer area corresponding to the target file with a specified size exists locally based on the identifier of the target file. If not, create a target buffer area corresponding to the target file in the local memory; the specified size is the size of a storage page in the NAND Flash medium of the electronic device;

[0013] The first processing module is configured to, if the size of the available storage space in the target buffer corresponding to the target file is smaller than the size of the target data to be written indicated by the data write request, before writing the target data to the NAND Flash medium, truncate the target data to obtain the first part of data and the second part of data, write the first part of data to the available storage space in the target buffer, where the size of the first part of data matches the size of the available storage space in the target buffer, then write the data in the target buffer to the NAND Flash medium, and clear the data in the target buffer, and write the second part of data to the available storage space in the target buffer based on the size of the available storage space in the target buffer; and when the size of the available storage space in the target buffer is a preset value, write the data in the target buffer to the NAND Flash medium, and clear the data in the target buffer;

[0014] The second processing module is configured to, if the size of the available storage space in the target buffer corresponding to the target file is greater than or equal to the size of the target data, before writing the target data to the NAND Flash medium, write the target data to the available storage space in the target buffer, and when the size of the available storage space in the target buffer is a preset value, write the data in the target buffer to the NAND Flash medium, and clear the data in the target buffer.

[0015] An embodiment of the present application further provides an electronic device, which includes:

[0016] A processor; and

[0017] A computer-readable storage medium, in which computer program instructions are stored, and when the computer program instructions are run by the processor, the processor is caused to execute the steps of the above method.

[0018] An embodiment of the present application further provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are run by the processor, the processor is caused to execute the steps in the above method.

[0019] As can be seen from the above technical solutions, in the embodiments of the present application, when a data writing request is received, first check whether there is a target buffer corresponding to the target file indicated by the request locally. If not, create a target buffer in the local memory, where the size of the target buffer is the same as the size of the storage page in the NAND Flash medium; then, when the size of the available storage space in the target buffer is less than the size of the target data to be written, truncate the target data to obtain a first part of the data and a second part of the data, and write them into the target buffer in batches. When the size of the available storage space in the target buffer is greater than or equal to the size of the target data, directly write the target data into the target buffer; based on this, when the size of the available storage space in the target buffer is a preset value, that is, when the target buffer is full, write the data in the target buffer into the NAND Flash medium, which can effectively avoid write amplification caused by the size of the target data to be written being less than the size of the storage page, thereby improving the PE life of the NAND Flash medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and form a part of this application, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.

[0021] Figure 1 It is a schematic flowchart of a method provided by an embodiment of the present application.

[0022] Figure 2 It is another schematic flowchart of a method provided by an embodiment of the present application.

[0023] Figure 3 It is yet another schematic flowchart of a method provided by an embodiment of the present application.

[0024] Figure 4 It is yet another schematic flowchart of a method provided by an embodiment of the present application.

[0025] Figure 5 It is a schematic structural diagram of a device provided by an embodiment of the present application.

[0026] Figure 6 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application and make the above-mentioned objects, features, and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0028] See Figure 1 , Figure 1The flowchart of the method provided by the embodiments of this application. This method is applied to an electronic device. As an embodiment, the electronic device can be, for example, an embedded device, a server, etc., and this embodiment does not specifically limit it. In this embodiment, this method can be applied to multiple scenarios such as the log data storage scenario, etc., and this embodiment does not specifically limit this. As Figure 1 shown, this process may include the following steps:

[0029] Step 101, receive a data write request; the data write request indicates the identifier of the target file.

[0030] In this embodiment, the target file may refer to the file to which the target data indicated by the data write request is to be written. Here, the file can be understood as the file organized and managed in the file system. The identifier of the target file can be, for example, the file name of the target file, etc., and this is not specifically limited here.

[0031] Step 102, check whether there is a target buffer area corresponding to the target file with a specified size in the local area based on the identifier of the target file. If not, create a target buffer area corresponding to the target file in the local memory; the specified size is the size of the storage page in the NAND Flash medium in the electronic device.

[0032] In this embodiment, as an embodiment, in this step, checking whether there is a target buffer area corresponding to the target file with a specified size in the local area based on the identifier of the target file can be specifically implemented as: searching from the file identifier - buffer area correspondence relationship recorded locally based on the identifier of the target file to find whether there is a buffer area corresponding to the identifier of the target file. If it exists, it is determined that there is a target buffer area corresponding to the target file in the local area. If it does not exist, it is determined that there is no target buffer area corresponding to the target file in the local area.

[0033] Based on this, when this embodiment determines that there is no target buffer area corresponding to the target file in the local area, it creates a target buffer area corresponding to the target file in the local memory for subsequent standby. Among them, the size of the target buffer area is the size of the storage page in the NAND Flash medium in the electronic device.

[0034] Among them, as described above, the above NAND Flash medium can be composed of multiple storage blocks, and each storage block can be composed of multiple storage pages. Here, the NAND Flash medium can be, for example, a Secure Digital (SD) card and a Solid State Disk (SSD), etc., and this embodiment does not specifically limit it.

[0035] In this embodiment, as an example, to avoid data loss in the target buffer after the power-off of the above-mentioned electronic device, the local memory in this embodiment can be a static random-access memory (SRAM) powered by a battery. In this way, it can be ensured that after the electronic device is powered off, the data in the SRAM powered by the battery, that is, the data in the target buffer, will not be lost, improving the reliability of data storage.

[0036] Step 103, if the size of the available storage space in the target buffer corresponding to the target file is smaller than the size of the target data to be written indicated by the data write request, then before writing the target data to the NAND Flash medium, truncate the target data to obtain a first part of data and a second part of data. Write the first part of data to the available storage space in the target buffer, where the size of the first part of data matches the size of the available storage space in the target buffer. After that, write the data in the target buffer to the NAND Flash medium, and clear the data in the target buffer. Based on the size of the available storage space in the target buffer, write the second part of data to the available storage space in the target buffer; and when the size of the available storage space in the target buffer is a preset value, write the data in the target buffer to the NAND Flash medium, and clear the data in the target buffer.

[0037] In this embodiment, if the size of the available storage space in the target buffer corresponding to the target file is smaller than the size of the target data to be written indicated by the data write request, it means that the available storage space in the target buffer is not enough to accommodate the target data. At this time, before writing the target data to the NAND Flash medium, the target data can be truncated to obtain a first part of data and a second part of data, so as to write the target data to the available storage space in the target buffer in batches. Specifically, for example, the first part of data can be written to the available storage space in the target buffer. Here, the size of the first part of data matches the size of the available storage space in the target buffer, that is, the size of the first part of data is the same as the size of the available storage space in the target buffer; after that, write the data in the target buffer to the NAND Flash medium, and clear the data in the target buffer to write the second part of data to the available storage space in the target buffer;

[0038] Based on this, when the size of the available storage space in the target buffer is a preset value, for example, the preset value can be 0, which means the target buffer is already full of data and there is no available storage space. As described above, the size of the target buffer is the same as the size of the storage page in the NAND Flash medium. Therefore, writing the data in the target buffer to the NAND Flash medium at this time is equivalent to writing data in the size of the entire storage page, which can avoid write amplification caused by the size of the target data to be written being smaller than the size of the storage page, thereby improving the PE life of the NAND Flash medium. Moreover, after writing the data in the target buffer to the NAND Flash medium, the data in the target buffer can be cleared to await subsequent data writing, realizing the reuse of the target buffer and thus avoiding waste of memory resources.

[0039] As for how to specifically write the data in the target buffer to the NAND Flash medium in this step, it will be described by examples below and will not be elaborated here for the time being.

[0040] Step 104, if the size of the available storage space in the target buffer corresponding to the target file is greater than or equal to the size of the target data, then before writing the target data to the NAND Flash medium, write the target data to the available storage space in the target buffer, and when the size of the available storage space in the target buffer is the preset value, write the data in the target buffer to the NAND Flash medium and clear the data in the target buffer.

[0041] In this embodiment, if the size of the available storage space in the target buffer corresponding to the target file is equal to the size of the target data, it means that the available storage space in the target buffer is just enough to accommodate the target data. At this time, before writing the target data to the NAND Flash medium, the target data can be directly written to the available storage space in the target buffer; after writing the target data to the available storage space in the target buffer, the target buffer is just full of data at this time, that is, the size of the available storage space in the target buffer is the preset value. Therefore, the data in the target buffer can be written to the NAND Flash medium and the data in the target buffer can be cleared to await subsequent continued use.

[0042] If the size of the available storage space in the target buffer corresponding to the target file is greater than the size of the target data, it means that the available storage space in the target buffer is sufficient to accommodate the target data. At this time, before writing the target data to the NAND Flash medium, the target data can be written to the available storage space in the target buffer, and when the size of the available storage space in the target buffer is the preset value subsequently, the data in the target buffer can be written to the NAND Flash medium.

[0043] As for how to write the second part of the data into the available storage space in the target buffer based on the size of the available storage space in the target buffer in the above steps, it is similar to how to write the target data into the target buffer based on the size of the available storage space in the target buffer.

[0044] Specifically, in this embodiment, as an example, if the size of the available storage space in the target buffer is smaller than the size of the second part of the data, it means that the available storage space in the target buffer is not enough to accommodate the second part of the data. At this time, the second part of the data can be truncated to obtain the third part of the data and the fourth part of the data, where the size of the third part of the data matches the size of the available storage space in the target buffer, and the third part of the data is written into the available storage space in the target buffer. At this time, as described above, the target buffer has been filled, so the data in the target buffer can be written into the NAND Flash medium, and the data in the target buffer is cleared. After that, the fourth part of the data is used as the second part of the data, and the step of writing the second part of the data into the available storage space in the target buffer based on the size of the available storage space in the target buffer is returned.

[0045] If the size of the available storage space in the target buffer is greater than or equal to the size of the second part of the data, as can be seen from the previous description, at this time, the available storage space in the target buffer is enough to accommodate the second part of the data. Therefore, the second part of the data can be directly written into the available storage space in the target buffer.

[0046] So far, the Figure 1 shown process is completed.

[0047] Through Figure 1 the shown process, it can be seen that in the embodiment of the present application, when a data write request is received, first check whether there is a target buffer corresponding to the target file indicated by the request locally. If not, create a target buffer in the local memory, where the size of the target buffer is the same as the size of the storage page in the NAND Flash medium; after that, when the size of the available storage space in the target buffer is smaller than the size of the target data to be written, truncate the target data to obtain the first part of the data and the second part of the data, and write them into the target buffer in batches. When the size of the available storage space in the target buffer is greater than or equal to the size of the target data, directly write the target data into the target buffer; based on this, when the size of the available storage space in the target buffer is a preset value, that is, when the target buffer is filled, write the data in the target buffer into the NAND Flash medium, which can effectively avoid write amplification caused by the size of the target data to be written being smaller than the size of the storage page, thereby improving the PE life of the NAND Flash medium.

[0048] The following describes how to write the data in the target buffer to the NAND Flash medium specifically as above:

[0049] In this embodiment, the files in the file system are stored depending on the index node (Inode). Among them, each file in the file system corresponds to an index node. The index node can be a data structure used to record the storage location of the file in the NAND Flash medium, as well as the metadata information of the file, etc. Here, the storage location of the file in the NAND Flash medium can be, for example, one of the storage blocks allocated to the file from the NAND Flash medium, and the storage time of the data in this storage block is the largest; the metadata information of the file can include, for example, the identifier, size, and creation time of the file, etc.

[0050] In this embodiment, as an example, the Inodes of each file in the file system are stored in a specified storage block in the NAND Flash medium. Specifically, for example, two storage blocks are selected in advance from the NAND Flash medium, and the Superblock and the backup Superblock are stored in the two storage blocks respectively, and the data content in the Superblock and the backup Superblock is the same. Here, the Superblock is a data structure that at least includes the metadata information of the file system (such as information such as the version, size, and number of files of the file system), and the index field Inode_ptr pointing to the storage location of the Inodes of each file in the file system. In the storage block containing the Superblock and the storage block containing the backup Superblock, the Inodes of each file in the file system are also included respectively; for example, one storage page in the storage block can store multiple Inodes. If the current storage page is already full, the next storage page can be found through the Next index pointing to the next storage page in the current storage page, so as to continue storing Inodes using the next storage page, and so on.

[0051] Based on this, as an example, when writing the data in the target buffer to the NAND Flash medium as above, the specific implementation may include: first obtaining the index field Inode_ptr from the NAND Flash medium to obtain the storage location of the Inodes of each file in the file system. Here, the storage location of the Inodes of each file in the file system can be, for example, the identifier of the storage page with the largest storage time of the data in the storage page storing the Inode;

[0052] After that, check whether there is an Inode that matches the identifier of the target file in the storage page indicated by the index field Inode_ptr. If there is an Inode that matches the identifier of the target file, when there is a free storage page in the storage block indicated by this Inode, write the data in the target buffer to this free storage page. When there is no free storage page in the storage block indicated by this Inode, that is, when it is full, find the next storage page through the Next index pointing to the next storage page in the last storage page of this storage block (this next storage page is a storage page in another storage block), and when this next storage page is a free storage page, write the data in the target buffer to this next storage page. When this next storage page is not a free storage page, find the next storage page through the Next index pointing to the next storage page in this next storage page, and so on;

[0053] If there is no Inode that matches the identifier of the target file, find the next storage page through the Next index pointing to the next storage page in the storage page indicated by the index field Inode_ptr to continue checking whether there is an Inode that matches the identifier of the target file, and so on;

[0054] Among them, if it is found that there is no free storage page in the storage block allocated to the target file, select a free storage block from the shared pool to allocate to the target file, and write the data in the target buffer to the free storage page in this free storage block.

[0055] So far, the description of how to write the data in the target buffer to the NAND Flash medium is completed. The following further describes the data processing process after writing the data in the target buffer to the NAND Flash medium:

[0056] In this embodiment, as an example, after writing the data in the target buffer to the NAND Flash medium, determine whether the target file needs to perform a rotation operation. If so, modify the identifier indicating the storage location of the target file in the NAND Flash medium, that is, the identifier of the storage location of the target file indicated by the above-mentioned Inode corresponding to the target file, from the block identifier of the first storage block to the block identifier of the second storage block. After that, erase the data in the first storage block and add the first storage block as a free storage block to the free pool for standby;

[0057] Among them, the storage blocks allocated to the target file include a first storage block and a second storage block; among the storage blocks allocated to the target file, the storage time of the data in the first storage block is the longest, and the storage time of the data in the second storage block is the closest to the storage time of the data in the first storage block. Here, the storage time of the data in any storage block may refer to the storage time corresponding to the data with the longest storage time in that storage block.

[0058] Exemplarily, assume that the storage blocks allocated to the target file are storage block 1, storage block 2, and storage block 3. The storage time of the data in storage block 1 is 3 days, the storage time of the data in storage block 2 is 2 days, and the storage time of the data in storage block 3 is 4 days. Then, among the storage blocks allocated to the target file, storage block 1 is the first storage block, and storage block 2 is the second storage block.

[0059] In this embodiment, as an example, the above determination of whether the target file needs to perform a rotation operation may be specifically implemented as follows: Check whether the size of the available storage space in the storage blocks allocated to the target file is a preset value. If so, determine that the target file needs to perform a rotation operation; if not, determine that the target file does not need to perform a rotation operation.

[0060] In this embodiment, in the scenario of storing log data, the longer the storage time of the log data, the lower its available value. When erasing data, data with low available value can be preferentially considered; based on this, in this embodiment, when there is no free storage page in the storage blocks allocated to the target file, the data in the storage block with the longest storage time among the storage blocks allocated to the target file, that is, the data with relatively low available value in the target file, is directly erased, and the erased storage block is added to the free block pool as a free storage block for standby. In this way, no additional data migration is required when erasing data, thereby effectively reducing the write amplification caused by data migration due to erasing data and improving the PE life of the NAND Flash medium.

[0061] The following describes the data processing flow after receiving a data read request:

[0062] In this embodiment, referring to Figure 2 the method flow chart shown, this process may include the following steps:

[0063] Step 201, receive a data read request, and the data read request indicates the identifier of the target file.

[0064] Step 202, read the data to be read indicated by the data read request from the NAND Flash medium.

[0065] In this embodiment, as an example, similar to writing the data in the target buffer into the NAND Flash medium as described above, in this step, the data to be read indicated by the data read request is read from the NAND Flash medium. In specific implementation, for example, it can be: first obtain the index field Inode_ptr from the NAND Flash medium to get the storage location of the Inodes pointing to each file in the file system; then, check whether there is an Inode matching the identifier of the target file in the storage page indicated by the index field Inode_ptr. If there is, search for and read the data to be read in each storage page in the storage block indicated by this Inode; if not, find the next storage page through the Next index pointing to the next storage page in the storage page indicated by the index field Inode_ptr, and continue to check whether there is an Inode matching the identifier of the target file in this next storage page, and so on, until an Inode matching the identifier of the target file is found, and then search for and read the data to be read in each storage page in the storage block indicated by this Inode.

[0066] In this embodiment, the data in each storage page in the storage block indicated by the Inode can be sorted in ascending order of the storage time of the data in bytes, such as the data of the first byte, the data of the second byte, the data of the third byte,..., the data of the nth byte (n>1). Among them, the storage time of the data of the first byte is greater than the storage time of the data of the second byte, the storage time of the data of the second byte is greater than the storage time of the data of the third byte, and so on, and so forth. Here, for any byte of data, this byte can be regarded as the offset of this data in the storage medium. For example, the offset of the data of the first byte in the storage medium is the first byte, the offset of the data of the second byte in the storage medium is the second byte, and so on.

[0067] Based on this, as an example, searching for and reading the data to be read in each storage page in the storage block indicated by the Inode can be implemented as follows in specific implementation: Based on the starting offset of the data to be read indicated by the data read request in the storage medium and the data volume of the data to be read, search for the data with the offset between the above starting offset and the sum of the starting offset and the data volume of the data to be read in each storage page in the storage block indicated by the Inode, and use this as the data to be read found in each storage page in the storage block indicated by the Inode.

[0068] Exemplarily, assume that the starting offset of the data to be read indicated by the data read request in the storage medium is 2, and the data volume of the data to be read is 10. Based on this, the data with an offset between 2 and 12 in each storage page of the storage block indicated by the Inode, that is, the data from the second byte to the twelfth byte in each storage page of the storage block indicated by the Inode, can be used as the data to be read found in each storage page of the storage block indicated by the Inode.

[0069] Step 203, if the size of the data read from the NAND Flash medium is smaller than the size of the data to be read, check whether the data to be read exists in the target buffer corresponding to the target file. If so, read the remaining unread data in the data to be read from the target buffer, and splice the data read from the NAND Flash medium and the data read from the target buffer to obtain the data to be read.

[0070] Based on the above description, in this embodiment, as an example, reading the remaining unread data in the data to be read from the target buffer can be specifically implemented as follows: first, based on the data volume of the data to be read indicated by the data read request and the data volume of the data read from the NAND Flash medium, determine the data volume of the remaining unread data; then, starting from the starting position of the target buffer, read data from the target buffer based on the data volume of the remaining unread data as the remaining unread data in the data to be read.

[0071] Exemplarily, assume that the data volume of the remaining unread data determined is 2 bytes, then start reading two bytes of data from the starting position of the target buffer as the remaining unread data in the data to be read.

[0072] Step 204, if the size of the data read from the NAND Flash medium is equal to the size of the data to be read, directly use the data read from the NAND Flash medium as the data to be read.

[0073] The data processing procedures in the case of the electronic device being powered off and powered on are described below respectively:

[0074] In this embodiment, as described above, since the data to be written is written into the NAND Flash medium after being written into the corresponding buffer first, in order to avoid data loss in the buffer, the buffer can be provided by SRAM powered by a battery. However, using SRAM powered by a battery to provide the buffer increases the device cost. In this embodiment, if cost savings are desired, the buffer is not provided by SRAM powered by a battery, but instead a Dynamic Random Access Memory (DRAM) is used to provide the buffer. However, since the data in the DRAM will be lost after power-off, in order to avoid data loss in the buffer, when the electronic device is about to shut down, the data in the buffer can be protected. Specifically, refer to Figure 3 the schematic diagram of the method flow shown, and the process may include the following steps:

[0075] Step 301, receive a shutdown instruction. If there is at least one buffer currently, create a temporary dump file, and allocate storage blocks from the NAND Flash medium for the temporary dump file based on the sizes of the data in each buffer.

[0076] In this embodiment, the total size of the storage blocks allocated from the NAND Flash medium for the temporary dump file is greater than or equal to the sizes of the data in each buffer, so as to ensure that all the data in each buffer can be written into the NAND Flash medium, thereby realizing the protection of the data in the buffer.

[0077] Step 302, splice the data in each buffer to obtain spliced data, write the spliced data into the storage blocks allocated for the temporary dump file, and then shut down the electronic device.

[0078] Based on the above description, after the electronic device is powered on, the data in the temporary dump file can be used to recover the data in each buffer that existed before the electronic device was shut down. Specifically, refer to Figure 4 the schematic diagram of the method flow shown, and the process may include the following steps:

[0079] Step 401, when it is detected that the electronic device is powered on, check whether there is a temporary dump file in the NAND Flash medium. If there is, for the data belonging to the same file in the temporary dump file, create a buffer corresponding to the file to which the data belonging to the same file belongs in the local memory, and write the data belonging to the same file into the created buffer.

[0080] Exemplarily, assume that the temporary dump file contains data belonging to File 1 and data belonging to File 2. Then, for the data belonging to File 1 in the temporary dump file, a buffer corresponding to File 1 is created in the local memory, and the data belonging to File 1 in the temporary dump file is written into the created buffer corresponding to File 1. Correspondingly, for the data belonging to File 2 in the temporary dump file, a buffer corresponding to File 2 is created in the local memory, and the data belonging to File 2 in the temporary dump file is written into the created buffer corresponding to File 2. In this way, all the data in the temporary dump file is written into the corresponding buffers.

[0081] Step 402, after writing the data in the temporary dump file into the corresponding buffer, delete the data in the storage block allocated to the temporary dump file.

[0082] In this embodiment, as an example, after deleting the data in the storage block allocated to the temporary dump file, delete the temporary dump file in the file system, and add the storage block allocated to the temporary dump file as a free storage block to the free pool for standby.

[0083] So far, the description of the method provided by the embodiments of this application is completed. Next, the device provided by the embodiments of this application will be described:

[0084] See Figure 5 , Figure 5 which is a schematic structural diagram of a data processing device provided by the embodiments of this application.

[0085] As Figure 5 shown, the data processing device 500 includes:

[0086] A receiving module 501, configured to receive a data writing request; the data writing request indicates the identifier of a target file;

[0087] An inspection module 502, configured to check whether there is a target buffer corresponding to the target file with a specified size in the local area based on the identifier of the target file. If not, create a target buffer corresponding to the target file in the local memory; the specified size is the size of a storage page in the NAND Flash medium of the electronic device;

[0088] The first processing module 503 is configured to, if the size of the available storage space in the target buffer area corresponding to the target file is smaller than the size of the target data to be written indicated by the data write request, truncate the target data into a first part of data and a second part of data before writing the target data into the NAND Flash medium, write the first part of data into the available storage space in the target buffer area, the size of the first part of data matching the size of the available storage space in the target buffer area, then write the data in the target buffer area into the NAND Flash medium, and clear the data in the target buffer area, and write the second part of data into the available storage space in the target buffer area based on the size of the available storage space in the target buffer area; and when the size of the available storage space in the target buffer area is a preset value, write the data in the target buffer area into the NAND Flash medium, and clear the data in the target buffer area.

[0089] The second processing module 504 is configured to, if the size of the available storage space in the target buffer area corresponding to the target file is greater than or equal to the size of the target data, write the target data into the available storage space in the target buffer area before writing the target data into the NAND Flash medium, and when the size of the available storage space in the target buffer area is a preset value, write the data in the target buffer area into the NAND Flash medium, and clear the data in the target buffer area.

[0090] As an embodiment, the device further includes:

[0091] The reading module is configured to receive a data read request, and the data read request indicates the identifier of the target file;

[0092] Read the data to be read indicated by the data read request from the NAND Flash medium; and when the size of the data read from the NAND Flash medium is smaller than the size of the data to be read, check whether the data to be read exists in the target buffer area corresponding to the target file, and if so, read the remaining unread data of the data to be read from the target buffer area, and splice the data read from the NAND Flash medium and the data read from the target buffer area to obtain the data to be read.

[0093] As an embodiment, the device further includes:

[0094] The determining module is configured to, after writing the data in the target buffer area into the NAND Flash medium, determine whether the target file needs to perform a rotation operation, and if so, modify the identifier indicating the storage location of the target file stored in the NAND Flash medium from the block identifier of the first storage block to the block identifier of the second storage block, and then erase the data in the first storage block.

[0095] Among them, the storage block is a storage block in the NAND Flash medium; the storage blocks allocated to the target file include a first storage block and a second storage block; the storage time of the data in the first storage block among the storage blocks allocated to the target file is the longest, and the storage time of the data in the second storage block is the closest to the storage time of the data in the first storage block.

[0096] As an embodiment, determining whether the target file needs to perform a rotation operation includes:

[0097] Checking whether the size of the available storage space in the storage blocks allocated to the target file is a preset value. If so, it is determined that the target file needs to perform a rotation operation; if not, it is determined that the target file does not need to perform a rotation operation.

[0098] As an embodiment, the device further includes:

[0099] A power-on module, configured to receive a shutdown instruction. If there is at least one buffer currently, create a temporary dump file, and allocate storage blocks from the NAND Flash medium for the temporary dump file based on the sizes of the data in each buffer;

[0100] Concatenate the data in each buffer to obtain concatenated data, write the concatenated data into the storage blocks allocated to the temporary dump file, and then shut down the electronic device.

[0101] As an embodiment, the device further includes:

[0102] A power-on module, configured to check whether there is a temporary dump file in the NAND Flash medium when detecting that the electronic device is powered on. If so, for the data belonging to the same file in the temporary dump file, create a buffer corresponding to the file to which the data belonging to the same file belongs in the local memory, and write the data belonging to the same file into the created buffer;

[0103] After writing the data in the temporary dump file into the corresponding buffer, delete the data in the storage blocks allocated to the temporary dump file.

[0104] Thus, the Figure 5 structural description of the shown device is completed.

[0105] For the specific implementation processes of the functions and roles of each module in the above device, please refer to the implementation processes of the corresponding steps in the above method in detail, which will not be elaborated here.

[0106] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the descriptions of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this application. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0107] Please refer to Figure 6 , which is a schematic diagram of the hardware structure of an electronic device provided by an exemplary embodiment of this application. The electronic device may include a processor 601, a communication interface 602, a computer-readable storage medium 603, and a communication bus 604. The processor 601, the communication interface 602, and the computer-readable storage medium 603 complete communication with each other through the communication bus 604. Among them, computer program instructions are stored on the computer-readable storage medium 603; the processor 601 can execute the steps of the method described in the above embodiments by executing the computer program instructions stored on the computer-readable storage medium 603. According to the actual functions of the electronic device, the electronic device may further include other hardware, which will not be elaborated here.

[0108] Correspondingly, an embodiment of this application also provides a computer-readable storage medium, on which several computer program instructions are stored. When the computer program instructions are executed by a processor, the methods disclosed in the above examples of this application can be implemented.

[0109] Exemplarily, the above computer-readable storage medium can be any electronic, magnetic, optical or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, the computer-readable storage medium can be: RAM (Radom Access Memory, random access memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or a combination thereof. The processor and the memory can be supplemented by or incorporated into dedicated logic circuits.

[0110] The above are only the preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A data processing method, characterized in that: The method is applied to an electronic device, and the method comprises: receiving a data write request; the data write request indicates an identifier of a target file; Based on the identifier of the target file, check whether there is a target cache area with a specified size corresponding to the target file locally, and if not, create a target cache area corresponding to the target file in the local memory; the specified size is the size of a storage page in a NAND Flash medium in the electronic device; If the size of the available storage space in the target cache area corresponding to the target file is smaller than the size of the target data to be written indicated by the data write request, before writing the target data to the NAND Flash medium, truncate the target data to obtain a first portion of data and a second portion of data, write the first portion of data to the available storage space in the target cache area, the size of the first portion of data matches the size of the available storage space in the target cache area, then write the data in the target cache area to the NAND Flash medium, clear the data in the target cache area, and write the second portion of data to the available storage space in the target cache area based on the size of the available storage space in the target cache area; and when the size of the available storage space in the target cache area is a preset value, write the data in the target cache area to the NAND Flash medium, and clear the data in the target cache area; If the size of the available storage space in the target cache area corresponding to the target file is greater than or equal to the size of the target data, the target data is written to the available storage space in the target cache area before the target data is written to the NAND Flash medium, and when the size of the available storage space in the target cache area is a preset value, the data in the target cache area is written to the NAND Flash medium, and the data in the target cache area is cleared.

2. The method according to claim 1, characterized in that The method further comprises: receiving a data read request, wherein the data read request indicates an identifier of a target file; The data to be read indicated by the data read request is read from the NAND Flash medium; and when the size of the data read from the NAND Flash medium is smaller than the size of the data to be read, checking whether the data to be read exists in the target cache area corresponding to the target file, and if so, reading the remaining unread data in the data to be read from the target cache area, and splicing the data read from the NAND Flash medium and the data read from the target cache area to obtain the data to be read.

3. The method according to claim 1, characterized in that After writing the data in the target cache area to the NAND Flash medium, the method further comprises: Determine whether the target file needs to be rotated, and if so, modify the identifier indicating the storage location of the target file stored in the NAND Flash medium from the block identifier of the first storage block to the block identifier of the second storage block, and then erase the data in the first storage block; Among them, the storage block is a storage block in the NAND Flash medium; the storage blocks allocated to the target file include the first storage block and the second storage block; among the storage blocks allocated to the target file, the storage time of the data in the first storage block is the longest, and the storage time of the data in the second storage block is closest to the storage time of the data in the first storage block.

4. The method according to claim 3, characterized in that The determining whether the target file needs to perform a rotation operation includes: Check whether the size of the available storage space in the storage block allocated to the target file is a preset value, if yes, determine that the target file needs to perform a rotation operation, if no, determine that the target file does not need to perform a rotation operation.

5. The method according to claim 1, characterized in that The method further comprises: Receiving a shutdown instruction, if there is currently at least one cache area, creating a temporary dump file, and allocating storage blocks for the temporary dump file from the NAND Flash medium based on the size of data in each cache area; The data in each buffer area are spliced ​​to obtain spliced ​​data, and the spliced ​​data is written to the storage block allocated to the temporary dump file, and then the electronic device is shut down.

6. The method according to claim 5, characterized in that The method further comprises: When it is detected that the electronic device is turned on, checking whether the temporary dump file exists in the NAND Flash medium, and if so, creating a cache area corresponding to the file to which the data belonging to the same file belongs in the local memory for the data belonging to the same file in the temporary dump file, and writing the data belonging to the same file into the created cache area; After writing the data in the temporary dump file into the corresponding buffer area, the data in the storage block allocated to the temporary dump file is deleted.

7. A data processing device, characterized in that: The device is applied to electronic equipment, and the device comprises: A receiving module, configured to receive a data writing request; the data writing request indicates an identifier of a target file; A checking module, configured to check whether there is a target cache area with a specified size corresponding to the target file locally based on the identifier of the target file, and if not, create a target cache area corresponding to the target file in a local memory; the specified size is the size of a storage page in a NAND Flash medium in the electronic device; A first processing module is used for, if the size of the available storage space in the target cache area corresponding to the target file is smaller than the size of the target data to be written indicated by the data write request, before writing the target data into the NAND Flash medium, truncating the target data to obtain a first portion of data and a second portion of data, writing the first portion of data into the available storage space in the target cache area, the size of the first portion of data matching the size of the available storage space in the target cache area, then writing the data in the target cache area into the NAND Flash medium, clearing the data in the target cache area, and writing the second portion of data into the available storage space in the target cache area based on the size of the available storage space in the target cache area; and when the size of the available storage space in the target cache area is a preset value, writing the data in the target cache area into the NAND Flash medium, and clearing the data in the target cache area; The second processing module is used to write the target data to the available storage space in the target cache area before writing the target data to the NAND Flash medium if the size of the available storage space in the target cache area corresponding to the target file is greater than or equal to the size of the target data, and when the size of the available storage space in the target cache area is a preset value, write the data in the target cache area to the NAND Flash medium and clear the data in the target cache area.

8. The device according to claim 7, characterized in that The device further comprises: a reading module, configured to receive a data reading request, wherein the data reading request indicates an identifier of a target file; read the data to be read indicated by the data reading request from the NAND Flash medium; and when the size of the data read from the NAND Flash medium is smaller than the size of the data to be read, check whether the data to be read exists in the target buffer area corresponding to the target file, and if so, read the remaining unread data in the data to be read from the target buffer area, and splice the data read from the NAND Flash medium and the data read from the target buffer area to obtain the data to be read; and / or, The device further comprises: a determination module, which is used to determine whether the target file needs to perform a rotation operation after writing the data in the target cache area to the NAND Flash medium, and if so, to modify the identifier indicating the storage location of the target file stored in the NAND Flash medium from the block identifier of the first storage block to the block identifier of the second storage block, and then erase the data in the first storage block; wherein the storage block is a storage block in the NAND Flash medium; the storage block allocated to the target file includes the first storage block and the second storage block; among the storage blocks allocated to the target file, the storage time of the data in the first storage block is the longest, and the storage time of the data in the second storage block is closest to the storage time of the data in the first storage block; and / or, The determining whether the target file needs to perform a rotation operation includes: checking whether the size of the available storage space in the storage block allocated to the target file is a preset value, if yes, determining that the target file needs to perform a rotation operation, if no, determining that the target file does not need to perform a rotation operation; and / or, The device further comprises: a shutdown module, configured to receive a shutdown instruction, create a temporary dump file if there is currently at least one cache area, and allocate a storage block from the NAND Flash medium to the temporary dump file based on the size of the data in each cache area; splice the data in each cache area to obtain spliced ​​data, and write the spliced ​​data to the storage block allocated to the temporary dump file, and then shut down the electronic device; and / or, The device further includes: a startup module, which is used to check whether the temporary dump file exists in the NAND Flash medium when it is detected that the electronic device is turned on. If so, for the data belonging to the same file in the temporary dump file, create a cache area corresponding to the file to which the data belonging to the same file belongs in the local memory, and write the data belonging to the same file into the created cache area; after writing the data in the temporary dump file into the corresponding cache area, delete the data in the storage block allocated to the temporary dump file.

9. An electronic device, characterized in that: The electronic device includes: Processor; and A computer-readable storage medium, wherein computer program instructions are stored in the computer-readable storage medium, and when the computer program instructions are executed by the processor, the processor executes the steps in any one of the methods of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, enable the processor to perform the steps of any one of the methods of claims 1 to 6.

Images