Data processing method and apparatus, and electronic device

Abstract

Embodiments of the present application provide a data processing method and device and electronic equipment. In the embodiments, when a data write request is received, it is first checked whether a target cache area corresponding to a target file indicated by the request exists locally. If not, a target cache area is created in the local memory, and the size of the target cache area is the same as the size of a storage page in a NAND Flash medium. Then, before writing the target data to be written into the NAND Flash medium, the target data is written into the target cache area based on the size of the available storage space of the target cache area. Based on this, when the size of the available storage space in the target cache area is a preset value, that is, the target cache area is full, the data in the target cache area is written into the NAND Flash medium. In this way, write amplification caused by the size of the target data to be written being smaller than the size of the storage page can be effectively avoided, thereby improving the PE life of the NAND Flash medium.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to data processing methods, apparatus and electronic devices. Background Technology

[0002] NAND flash memory is a non-volatile storage medium with high sequential write speed, large storage density, and long PE (Program / Erase) lifetime, making it widely used in large-capacity data storage. PE stands for Program / Erase. The structure of NAND flash memory consists of multiple blocks (referred to as storage blocks), and each block contains multiple pages (referred to as storage pages).

[0003] In log data storage scenarios, the smallest write unit in Nand flash media is typically a page, and the smallest erase unit is typically a block. Therefore, if the amount of data to be written in a write request is less than the size of a page, the write operation will occupy the entire page's write capacity, resulting in write amplification. If all available blocks in the Nand flash media are occupied, a rotate operation will be triggered, deleting old data files. Deletion usually involves data migration, such as migrating valid data from the block to be deleted to other blocks before erasing the data in the block to be deleted. This results in the actual amount of data written exceeding the amount of data to be written in the write request, also causing write amplification. Write amplification reduces the PE lifetime of the Nand flash media. Write amplification refers to the phenomenon where the actual amount of data written to the storage medium exceeds the amount of data expected by the write request. Therefore, effectively reducing write amplification to improve the PE lifetime of Nand flash media is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] In view of this, this application provides a data processing method, apparatus, and electronic equipment to reduce write amplification and improve the PE lifetime of Nand flash media.

[0005] This application also provides a data processing method applied to an electronic device, the method comprising:

[0006] Receive a data write request; the data write request indicates the identifier of the target file;

[0007] Based on the identifier of the target file, check if a target cache of the specified size exists locally. If not, create the target cache corresponding to the target file in local memory; the specified size is the size of a storage page in the NAND Flash medium of the electronic device.

[0008] If the available storage space in the target cache corresponding to the target file is smaller than the size of the target data to be written as indicated by the data write request, then before writing the target data to the NAND Flash medium, the target data is truncated to obtain a first part of data and a second part of data. The first part of data is written to the available storage space in the target cache, and the size of the first part of data matches the size of the available storage space in the target cache. Then, the data in the target cache is written to the NAND Flash medium, and the data in the target cache is cleared. Based on the size of the available storage space in the target cache, the second part of data is written to the available storage space in the target cache. When the size of the available storage space in the target cache is a preset value, the data in the target cache is written to the NAND Flash medium, and the data in the target cache is cleared.

[0009] If the available storage space in the target cache corresponding to the target file is greater than or equal to the size of the target data, the target data is written to the available storage space in the target cache before writing the target data to the NAND Flash medium. When the available storage space in the target cache is the preset value, the data in the target cache is written to the NAND Flash medium and the data in the target cache is cleared.

[0010] This application embodiment also provides a data processing apparatus, which is applied to an electronic device, and the apparatus includes:

[0011] The receiving module is used to receive data write requests; the data write request indicates the identifier of the target file.

[0012] The inspection module is used to check whether a target cache of a specified size exists locally based on the identifier of the target file. If not, the target cache corresponding to the target file is created in local memory; the specified size is the size of a storage page in the NAND Flash medium of the electronic device.

[0013] The first processing module is configured to, if the available storage space in the target cache corresponding to the target file is less than the size of the target data to be written as indicated by the data write request, truncate the target data to obtain a first part of data and a second part of data before writing the target data to the NAND Flash medium, write the first part of data to the available storage space in the target cache, the size of the first part of data matching the size of the available storage space in the target cache, then write the data in the target cache to the NAND Flash medium and clear the data in the target cache, write the second part of data to the available storage space in the target cache based on the size of the available storage space in the target cache; and when the size of the available storage space in the target cache is a preset value, write the data in the target cache to the NAND Flash medium and clear the data in the target cache.

[0014] The second processing module is used to write the target data into the available storage space in the target cache before writing the target data into the NAND Flash medium if the available storage space in the target cache corresponding to the target file is greater than or equal to the size of the target data. When the available storage space in the target cache is a preset value, the data in the target cache is written into the NAND Flash medium and the data in the target cache is cleared.

[0015] This application also provides an electronic device, which includes:

[0016] Processor; and

[0017] A computer-readable storage medium storing computer program instructions that, when executed by a processor, cause the processor to perform the steps of the method described above.

[0018] This application also provides a computer-readable storage medium storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the above method.

[0019] As can be seen from the above technical solutions, in this embodiment, when a data write request is received, the system first checks whether a target cache corresponding to the target file indicated by the request exists locally. If it does not exist, a target cache is created in local memory, wherein the size of the target cache is the same as the size of a storage page in the NAND Flash medium. Then, when the available storage space in the target cache is less than the size of the target data to be written, the target data is truncated to obtain a first part of data and a second part of data, which are written to the target cache in batches. When the available storage space in the target cache is greater than or equal to the size of the target data, the target data is directly written to the target cache. Based on this, when the available storage space in the target cache is a preset value, that is, when the target cache is full, the data in the target cache is written to the NAND Flash medium. This can effectively avoid write amplification caused by the size of the target data to be written being less than the size of a storage page, thereby improving the PE lifespan of the NAND Flash medium. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this application, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] Figure 1 This is a schematic diagram of a method provided in an embodiment of this application.

[0022] Figure 2 This is a schematic diagram of another method flow provided for an embodiment of this application.

[0023] Figure 3 This is a schematic diagram of another method flow provided in an embodiment of this application.

[0024] Figure 4 This is a schematic diagram of another method flow provided in an embodiment of this application.

[0025] Figure 5 This is a schematic diagram of the device structure provided in the embodiments of this application.

[0026] Figure 6 This is a schematic diagram of the electronic device structure provided in an embodiment of this application. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0028] See Figure 1 , Figure 1This is a flowchart illustrating a method provided in an embodiment of this application. The method is applied to an electronic device. As one embodiment, the electronic device may be an embedded device, a server, etc., and this embodiment is not specifically limited thereto. In this embodiment, the method can be applied to various scenarios, such as log data storage scenarios, and this embodiment is not specifically limited thereto. Figure 1 As shown, the process may include the following steps:

[0029] Step 101: Receive a data write request; the data write request indicates the identifier of the target file.

[0030] In this embodiment, the target file can refer to the file to which the target data to be written, as indicated by the data write request, is to be written. Here, a file can be understood as a file organized and managed in a file system. The identifier of the target file can be, for example, the filename of the target file, etc., but this is not specifically limited here.

[0031] Step 102: Based on the identifier of the target file, check if a target cache of the specified size exists locally. If not, create the target cache corresponding to the target file in local memory; the specified size is the size of a storage page in the NAND Flash medium of the electronic device.

[0032] In this embodiment, as an example, this step checks whether a target cache area of ​​a specified size exists locally based on the target file's identifier. In a specific implementation, for example, it can be: based on the target file's identifier, search the locally recorded file identifier-cache area correspondence to see if there is a cache area corresponding to the target file's identifier. If it exists, it is determined that a target cache area corresponding to the target file exists locally; if it does not exist, it is determined that a target cache area corresponding to the target file does not exist locally.

[0033] Therefore, in this embodiment, when it is determined that a target cache corresponding to the target file does not exist locally, a target cache corresponding to the target file is created in local memory for later use. The size of the target cache is equal to the size of a storage page in the NAND Flash medium of the electronic device.

[0034] As described above, the NAND Flash medium can be composed of multiple storage blocks, and each storage block can be composed of multiple storage pages. Here, the NAND Flash medium can be, for example, a Secure Digital (SD) card or a Solid State Disk (SSD), etc., and this embodiment is not specifically limited to these.

[0035] In this embodiment, as an example, to avoid data loss in the target cache area after the electronic device loses power, the local memory in this embodiment can be a battery-powered static random-access memory (SRAM). This ensures that the data in the battery-powered SRAM, i.e. the data in the target cache area, is not lost after the electronic device loses power, thus improving the reliability of data storage.

[0036] Step 103: If the available storage space in the target cache corresponding to the target file is less than the size of the target data to be written as indicated by the data write request, then before writing the target data to the NAND Flash medium, the target data is truncated to obtain a first part of data and a second part of data. The first part of data is written to the available storage space in the target cache, and the size of the first part of data matches the size of the available storage space in the target cache. Then, the data in the target cache is written to the NAND Flash medium, and the data in the target cache is cleared. The second part of data is written to the available storage space in the target cache based on the size of the available storage space in the target cache. When the size of the available storage space in the target cache is a preset value, the data in the target cache is written to the NAND Flash medium, and the data in the target cache is cleared.

[0037] In this embodiment, if the available storage space in the target cache corresponding to the target file is smaller than the size of the target data to be written as indicated by the data write request, it means that the available storage space in the target cache is insufficient to accommodate the target data. In this case, before writing the target data to the NAND Flash medium, the target data can be truncated to obtain a first part of data and a second part of data, so as to write the target data to the available storage space in the target cache in batches. Specifically, the first part of data can be written to the available storage space in the target cache. Here, the size of the first part of data matches the size of the available storage space in the target cache, that is, the size of the first part of data is the same as the size of the available storage space in the target cache. After that, the data in the target cache is written to the NAND Flash medium, and the data in the target cache is cleared so that the second part of data can be written to the available storage space in the target cache.

[0038] Based on this, when the available storage space in the target cache is a preset value (e.g., 0), it indicates that the target cache is full and there is no available storage space. As described above, the size of the target cache is the same as the size of a storage page in the NAND Flash medium. Therefore, writing data from the target cache to the NAND Flash medium is equivalent to writing data at the size of an entire storage page. This avoids write amplification caused by the target data being smaller than the storage page size, thus improving the PE lifespan of the NAND Flash medium. Furthermore, after writing data from the target cache to the NAND Flash medium, the data in the target cache can be cleared for subsequent data writing, enabling the reuse of the target cache and avoiding waste of memory resources.

[0039] The specific steps for writing data from the target cache to the NAND Flash medium will be described with examples below, and will not be elaborated here.

[0040] Step 104: If the available storage space in the target cache corresponding to the target file is greater than or equal to the size of the target data, then before writing the target data to the NAND Flash medium, write the target data to the available storage space in the target cache. When the size of the available storage space in the target cache is a preset value, write the data in the target cache to the NAND Flash medium and clear the data in the target cache.

[0041] In this embodiment, if the available storage space in the target cache corresponding to the target file is equal to the size of the target data, it means that the available storage space in the target cache is just enough to hold the target data. At this time, before writing the target data to the NAND Flash medium, the target data can be directly written to the available storage space in the target cache. After the target data is written to the available storage space in the target cache, the target cache is just full of data, that is, the size of the available storage space in the target cache is the preset value. Therefore, the data in the target cache can be written to the NAND Flash medium, and the data in the target cache can be cleared for subsequent use.

[0042] If the available storage space in the target cache corresponding to the target file is greater than the size of the target data, it means that the available storage space in the target cache is sufficient to accommodate the target data. In this case, before writing the target data to the NAND Flash medium, the target data can be written to the available storage space in the target cache, and the data in the target cache can be written to the NAND Flash medium when the size of the available storage space in the target cache is the preset value.

[0043] As for how the second part of the data is written to the available storage space in the target cache based on the size of the available storage space in the target cache, the process is similar to how the target data is written to the target cache based on the size of the available storage space in the target cache.

[0044] Specifically, in this embodiment, if the available storage space in the target cache is smaller than the size of the second part of the data, it means that the available storage space in the target cache is insufficient to accommodate the second part of the data. In this case, the second part of the data can be truncated to obtain the third part of the data and the fourth part of the data. The size of the third part of the data matches the size of the available storage space in the target cache, and the third part of the data is written into the available storage space in the target cache. As described above, the target cache is now full. Therefore, the data in the target cache can be written into the NAND Flash medium, and the data in the target cache can be cleared. Then, the fourth part of the data is used as the second part of the data, and the process returns to the step of writing the second part of the data into the available storage space in the target cache based on the size of the available storage space in the target cache.

[0045] If the available storage space in the target cache is greater than or equal to the size of the second part of the data, then as described above, the available storage space in the target cache is sufficient to accommodate the second part of the data. Therefore, the second part of the data can be directly written into the available storage space in the target cache.

[0046] This concludes the process. Figure 1 The process is shown below.

[0047] pass Figure 1 As shown in the process, in this embodiment, when a data write request is received, the system first checks whether a target cache corresponding to the target file indicated by the request exists locally. If it does not exist, a target cache is created in local memory, wherein the size of the target cache is the same as the size of a storage page in the NAND Flash medium. Then, when the available storage space in the target cache is less than the size of the target data to be written, the target data is truncated to obtain a first part of data and a second part of data, which are written to the target cache in batches. When the available storage space in the target cache is greater than or equal to the size of the target data, the target data is directly written to the target cache. Based on this, when the available storage space in the target cache is a preset value, that is, when the target cache is full, the data in the target cache is written to the NAND Flash medium. This can effectively avoid write amplification caused by the size of the target data to be written being less than the size of a storage page, thereby improving the PE lifespan of the NAND Flash medium.

[0048] The following describes the specific steps for writing data from the target buffer to the NAND Flash medium:

[0049] In this embodiment, files in the file system rely on inodes for storage. Each file in the file system corresponds to one inode. An inode can be a data structure used to record the file's storage location in the NAND Flash medium, as well as the file's metadata information. Here, the file's storage location in the NAND Flash medium can be, for example, a storage block allocated to the file from the NAND Flash medium, where the data has been stored for the longest time; the file's metadata information can include the file's identifier, size, and creation time.

[0050] In this embodiment, as one example, the inodes of each file in the file system are stored in designated storage blocks in the NAND Flash medium. Specifically, two storage blocks are pre-selected from the NAND Flash medium, and a superblock and a backup superblock are stored in the two storage blocks respectively. The data content in the superblock and the backup superblock is the same. Here, the superblock is a data structure that includes at least the file system's metadata information (such as the file system version, size, number of files, etc.) and an index field Inode_ptr pointing to the storage location of the inodes of each file in the file system. The storage block containing the superblock and the storage block containing the backup superblock also include the inodes of each file in the file system. If a storage page in a storage block can store multiple inodes, if the current storage page is full, the next storage page can be found through the Next index pointing to the next storage page in the current storage page, so that the next storage page can be used to continue storing inodes, and so on.

[0051] Based on this, as an example, the above-mentioned writing of data in the target cache to the NAND Flash medium may include, for example, obtaining the index field Inode_ptr from the NAND Flash medium to obtain the storage location of the Inode of each file in the file system. Here, the storage location of the Inode of each file in the file system may be, for example, the identifier of the storage page with the longest storage time in the storage page storing the Inode.

[0052] Next, the system searches the storage page indicated by the index field Inode_ptr to see if there is an Inode that matches the identifier of the target file. If an Inode that matches the identifier of the target file exists, the data in the target cache is written to the free storage page if there is a free storage page in the storage block indicated by the Inode. If there is no free storage page in the storage block indicated by the Inode, that is, it is full, the system finds the next storage page (which is a storage page in another storage block) through the Next index pointing to the next storage page in the last storage page of the storage block. If the next storage page is a free storage page, the data in the target cache is written to the next storage page. If the next storage page is not a free storage page, the system finds the next storage page through the Next index pointing to the next storage page in the next storage page, and so on.

[0053] If no Inode matching the identifier of the target file is found, the next storage page is found by using the Next index pointing to the next storage page in the storage page indicated by the index field Inode_ptr, and the search continues to find whether there is an Inode matching the identifier of the target file, and so on.

[0054] If no free storage pages are found in the storage block to which the target file is allocated, a free storage block is selected from the shared pool to be allocated to the target file, and the data in the target cache is written to the free storage pages in the free storage block.

[0055] This concludes the description of how to write data from the target buffer to the NAND Flash medium. The following section further describes the data processing procedures following the writing of data from the target buffer to the NAND Flash medium:

[0056] In this embodiment, as an example, after writing the data in the target cache to the NAND Flash medium, it is determined whether the target file needs to be rotated. If so, the identifier indicating the storage location of the target file, which is already stored in the NAND Flash medium, i.e. the identifier indicating the storage location of the target file indicated by the Inode corresponding to the target file, is changed from the block identifier of the first storage block to the block identifier of the second storage block. Then, the data in the first storage block is erased, and the first storage block is added to the free pool as a free storage block for later use.

[0057] The target file is allocated storage blocks, including a first storage block and a second storage block. The data in the first storage block has the longest storage time, and the data in the second storage block has a storage time closest to that of the data in the first storage block. Here, the storage time of data in any storage block can refer to the storage time corresponding to the longest-stored data in that block.

[0058] For example, assuming the target file is allocated storage blocks 1, 2 and 3, the data in storage block 1 is stored for 3 days, the data in storage block 2 is stored for 2 days and the data in storage block 3 is stored for 4 days, then storage block 1 is the first storage block and storage block 2 is the second storage block.

[0059] In this embodiment, as an example, the above-mentioned determination of whether the target file needs to perform a rotation operation can be implemented as follows: check whether the size of the available storage space in the storage block allocated to the target file is a preset value. If it is, then determine that the target file needs to perform a rotation operation; if not, then determine that the target file does not need to perform a rotation operation.

[0060] In this embodiment, since the longer the log data is stored in the log data storage scenario, the lower its usability, data with low usability can be prioritized when erasing data. Based on this, this embodiment directly erases the data in the storage block with the longest storage time in the storage block allocated to the target file, i.e., the data with low usability in the target file, when there are no free storage pages in the storage block allocated to the target file. The erased storage block is then added to the free block pool as a free storage block for later use. In this way, no additional data migration is required when erasing data, which can effectively reduce write amplification caused by data migration due to data erasure and improve the PE life of the NAND Flash medium.

[0061] The following describes the data processing flow after receiving a data read request:

[0062] In this embodiment, see Figure 2 The method flowchart shown may include the following steps:

[0063] Step 201: Receive a data read request, which indicates the identifier of the target file.

[0064] Step 202: Read the data to be read from the NAND Flash medium as indicated by the data read request.

[0065] In this embodiment, as an example, similar to writing data from the target cache to the NAND Flash medium as described above, the step of reading the data to be read from the NAND Flash medium according to the data read request can be implemented as follows: First, obtain the index field Inode_ptr from the NAND Flash medium to obtain the storage location of the Inodes pointing to each file in the file system; then, search for an Inode that matches the identifier of the target file in the storage page indicated by the index field Inode_ptr. If it exists, search for the data to be read in each storage page of the storage block indicated by the Inode and read it; if it does not exist, find the next storage page through the Next index pointing to the next storage page in the storage page indicated by the index field Inode_ptr, and continue to search for an Inode that matches the identifier of the target file in the next storage page, and so on, until an Inode that matches the identifier of the target file is found, and search for the data to be read in each storage page of the storage block indicated by the Inode and read it.

[0066] In this embodiment, the data in each storage page of the storage block indicated by the Inode can be ordered byte by byte according to the storage time, such as the first byte, the second byte, the third byte, ..., the nth byte (n is greater than 1), where the storage time of the first byte is greater than that of the second byte, the storage time of the second byte is greater than that of the third byte, and so on. Here, for any byte of data, that byte can be considered as the offset of that data in the storage medium. For example, the offset of the first byte of data in the storage medium is the first byte, the offset of the second byte of data in the storage medium is the second byte, and so on.

[0067] Based on this, as an example, the above-mentioned search for and reading of the data to be read in each storage page of the storage block indicated by the Inode can be implemented as follows: based on the starting offset of the data to be read in the storage medium indicated by the data read request and the amount of data to be read, the data whose offset is between the starting offset and the sum of the starting offset and the amount of data to be read is searched in each storage page of the storage block indicated by the Inode, and this data is used as the data to be read found in each storage page of the storage block indicated by the Inode.

[0068] For example, suppose the starting offset of the data to be read in the storage medium indicated by the data read request is 2, and the amount of data to be read is 10. Based on this, the data with offsets between 2 and 12 in each storage page of the storage block indicated by the Inode, that is, the data from the 2nd byte to the 12th byte in each storage page of the storage block indicated by the Inode, can be used as the data to be read found in each storage page of the storage block indicated by the Inode.

[0069] Step 203: If the size of the data read from the NAND Flash medium is smaller than the size of the data to be read, check whether the data to be read exists in the target cache corresponding to the target file. If so, read the remaining unread data from the data to be read from the target cache, and concatenate the data read from the NAND Flash medium and the data read from the target cache to obtain the data to be read.

[0070] Based on the above description, in this embodiment, as an example, reading the remaining unread data from the target cache can be implemented as follows: first, based on the amount of data to be read indicated by the data read request and the amount of data read from the NAND Flash medium, the amount of remaining unread data is determined; then, starting from the beginning position of the target cache, data is read from the target cache based on the amount of remaining unread data, as the remaining unread data in the data to be read.

[0071] For example, assuming that the amount of remaining unread data is determined to be 2 bytes, then two bytes of data are read from the beginning of the target buffer as the remaining unread data in the data to be read.

[0072] Step 204: If the size of the data read from the NAND Flash medium is equal to the size of the data to be read, then the data read from the NAND Flash medium is directly used as the data to be read.

[0073] The data processing flow is described below for both the powered-off and powered-on states of the electronic device:

[0074] In this embodiment, as described above, since the data to be written is first written to the NAND Flash medium using a corresponding buffer, the buffer can be provided by a battery-powered SRAM to avoid data loss. However, using a battery-powered SRAM to provide the buffer increases device cost. In this embodiment, to save costs, the buffer is not provided by a battery-powered SRAM, but by Dynamic Random Access Memory (DRAM). However, since DRAM data is lost when power is off, the data in the buffer can be protected when the electronic device is about to be powered off to prevent data loss. Specifically, see [link to relevant documentation]. Figure 3 The flowchart shown may include the following steps:

[0075] Step 301: Receive a shutdown command. If at least one cache exists, create a temporary dump file and allocate storage blocks from the NAND Flash medium for the temporary dump file based on the size of the data in each cache.

[0076] In this embodiment, the total size of the storage blocks allocated from the NAND Flash medium for the temporary dump file is greater than or equal to the size of the data in each cache area, so as to ensure that all the data in each cache area can be written to the NAND Flash medium, thereby protecting the data in the cache area.

[0077] Step 302: The data in each buffer area are concatenated to obtain concatenated data, and the concatenated data is written to the storage block allocated to the temporary dump file. Then, the electronic device is powered off.

[0078] Based on the above description, after an electronic device is powered on, data from the temporary dump file can be used to recover data from the various cache areas that existed before the device was powered off. For details, see [link to documentation]. Figure 4 The flowchart shown may include the following steps:

[0079] Step 401: When the electronic device is detected to be powered on, check whether there is a temporary dump file in the NAND Flash medium. If there is, for the data belonging to the same file in the temporary dump file, create a cache area in the local memory corresponding to the file to which the data belongs, and write the data belonging to the same file to the created cache area.

[0080] For example, assuming the temporary dump file contains data belonging to file 1 and data belonging to file 2, then for the data belonging to file 1 in the temporary dump file, a cache corresponding to file 1 is created in local memory, and the data belonging to file 1 in the temporary dump file is written to the created cache corresponding to file 1; correspondingly, for the data belonging to file 2 in the temporary dump file, a cache corresponding to file 2 is created in local memory, and the data belonging to file 2 in the temporary dump file is written to the created cache corresponding to file 2. This completes the process of writing all the data in the temporary dump file to the corresponding cache.

[0081] Step 402: After writing the data in the temporary dump file to the corresponding cache area, delete the data in the storage block to which the temporary dump file was allocated.

[0082] In this embodiment, as an example, after deleting the data in the storage block to which the temporary dump file was allocated, the temporary dump file in the file system is deleted, and the storage block to which the temporary dump file was allocated is added to the free pool as a free storage block for later use.

[0083] This concludes the description of the method provided in the embodiments of this application. The apparatus provided in the embodiments of this application will now be described:

[0084] See Figure 5 , Figure 5 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application.

[0085] like Figure 5 As shown, the data processing apparatus 500 includes:

[0086] The receiving module 501 is used to receive a data write request; the data write request indicates the identifier of the target file.

[0087] The inspection module 502 is used to check whether a target cache of a specified size exists locally based on the identifier of the target file. If not, the target cache corresponding to the target file is created in local memory. The specified size is the size of a storage page in the NAND Flash medium of the electronic device.

[0088] The first processing module 503 is configured to, if the available storage space in the target cache corresponding to the target file is less than the size of the target data to be written as indicated by the data write request, truncate the target data to obtain a first part of data and a second part of data before writing the target data to the NAND Flash medium, write the first part of data to the available storage space in the target cache, the size of the first part of data matching the size of the available storage space in the target cache, then write the data in the target cache to the NAND Flash medium and clear the data in the target cache, write the second part of data to the available storage space in the target cache based on the size of the available storage space in the target cache; and when the size of the available storage space in the target cache is a preset value, write the data in the target cache to the NAND Flash medium and clear the data in the target cache.

[0089] The second processing module 504 is used to write the target data into the available storage space in the target cache before writing the target data into the NAND Flash medium if the available storage space in the target cache corresponding to the target file is greater than or equal to the size of the target data, and to write the data in the target cache into the NAND Flash medium and clear the data in the target cache when the size of the available storage space in the target cache is a preset value.

[0090] As one embodiment, the device further includes:

[0091] The reading module is used to receive data reading requests, which indicate the identifier of the target file;

[0092] The system reads the data requested from the NAND Flash medium. If the size of the data read from the NAND Flash medium is smaller than the size of the data to be read, it checks whether the data to be read exists in the target cache corresponding to the target file. If so, it reads the remaining unread data from the target cache and concatenates the data read from the NAND Flash medium and the data read from the target cache to obtain the data to be read.

[0093] As one embodiment, the device further includes:

[0094] The determination module is used to determine whether the target file needs to be rotated after the data in the target cache is written to the NAND Flash medium. If so, the identifier indicating the storage location of the target file stored in the NAND Flash medium is changed from the block identifier of the first storage block to the block identifier of the second storage block, and then the data in the first storage block is erased.

[0095] Among them, the storage block is the storage block in the NAND Flash medium; the storage blocks allocated to the target file include the first storage block and the second storage block; the storage time of the data in the first storage block is the longest, and the storage time of the data in the second storage block is the closest to the storage time of the data in the first storage block.

[0096] As an example, determining whether a target file needs to undergo a rotation operation includes:

[0097] Check if the available storage space in the allocated storage block of the target file is the preset value. If it is, determine that the target file needs to perform a rotation operation; otherwise, determine that the target file does not need to perform a rotation operation.

[0098] As one embodiment, the device further includes:

[0099] The power-on module is used to receive power-off commands. If at least one buffer exists, it creates a temporary dump file and allocates storage blocks from the NAND Flash medium for the temporary dump file based on the size of the data in each buffer.

[0100] The data in each buffer is concatenated to obtain concatenated data, which is then written to the storage block allocated to the temporary dump file. After that, the electronic device is powered off.

[0101] As one embodiment, the device further includes:

[0102] The power-on module is used to check whether there is a temporary dump file in the NAND Flash medium when the electronic device is detected to be powered on. If there is, for the data belonging to the same file in the temporary dump file, a cache area corresponding to the file to which the data belongs to the same file is created in the local memory, and the data belonging to the same file is written to the created cache area.

[0103] After writing the data from the temporary dump file to the corresponding cache, delete the data from the storage block allocated to the temporary dump file.

[0104] This concludes the process. Figure 5 Structural description of the device shown.

[0105] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0106] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0107] Please see Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an exemplary embodiment of this application. The electronic device may include a processor 601, a communication interface 602, a computer-readable storage medium 603, and a communication bus 604. The processor 601, the communication interface 602, and the computer-readable storage medium 603 communicate with each other via the communication bus 604. The computer-readable storage medium 603 stores computer program instructions; the processor 601 can execute the steps of the method described in the above embodiments by executing the computer program instructions stored on the computer-readable storage medium 603. Depending on the actual function of the electronic device, other hardware may also be included, which will not be elaborated further.

[0108] Correspondingly, embodiments of this application also provide a computer-readable storage medium storing a plurality of computer program instructions, which, when executed by a processor, can implement the methods disclosed in the above examples of this application.

[0109] For example, the aforementioned computer-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For instance, computer-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof. The processor and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0110] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data processing method, characterized in that, The method is applied to an electronic device, and the method includes: Receive a data write request; the data write request indicates the identifier of the target file; Based on the identifier of the target file, check if a target cache area of ​​a specified size corresponding to the target file exists locally. If not, create the target cache area corresponding to the target file in local memory. The specified size is the size of a storage page in the NAND Flash medium of the electronic device. If the available storage space in the target cache corresponding to the target file is smaller than the size of the target data to be written as indicated by the data write request, then before writing the target data to the NAND Flash medium, the target data is truncated to obtain a first part of data and a second part of data. The first part of data is written to the available storage space in the target cache, and the size of the first part of data matches the size of the available storage space in the target cache. Then, the data in the target cache is written to the NAND Flash medium, and the data in the target cache is cleared. Based on the size of the available storage space in the target cache, the second part of data is written to the available storage space in the target cache. When the size of the available storage space in the target cache is a preset value, the data in the target cache is written to the NAND Flash medium, and the data in the target cache is cleared. If the available storage space in the target cache corresponding to the target file is greater than or equal to the size of the target data, then before writing the target data to the NAND Flash medium, the target data is written to the available storage space in the target cache. When the size of the available storage space in the target cache is a preset value, the data in the target cache is written to the NAND Flash medium, and the data in the target cache is cleared.

2. The method according to claim 1, characterized in that, The method further includes: Receive a data read request, wherein the data read request indicates the identifier of the target file; The system reads the data to be read as indicated by the data read request from the NAND Flash medium; and if the size of the data read from the NAND Flash medium is less than the size of the data to be read, it checks whether the data to be read exists in the target cache corresponding to the target file. If so, it reads the remaining unread data from the data to be read from the target cache, and concatenates the data read from the NAND Flash medium and the data read from the target cache to obtain the data to be read.

3. The method according to claim 1, characterized in that, After writing the data in the target cache to the NAND Flash medium, the method further includes: Determine whether the target file needs to be rotated. If so, change the identifier indicating the storage location of the target file stored in the NAND Flash medium from the block identifier of the first storage block to the block identifier of the second storage block, and then erase the data in the first storage block. Wherein, the storage block is a storage block in the NAND Flash medium; the storage blocks to which the target file is allocated include the first storage block and the second storage block; among the storage blocks to which the target file is allocated, the data in the first storage block has the longest storage time, and the data in the second storage block has the closest storage time to the data in the first storage block.

4. The method according to claim 3, characterized in that, Determining whether the target file needs to undergo a rotation operation includes: Check whether the available storage space in the storage block allocated to the target file is a preset value. If it is, determine that the target file needs to perform a rotation operation; otherwise, determine that the target file does not need to perform a rotation operation.

5. The method according to claim 1, characterized in that, The method further includes: Upon receiving a shutdown command, if at least one cache exists, a temporary dump file is created, and storage blocks are allocated from the NAND Flash medium for the temporary dump file based on the size of the data in each cache. The data in each cache area is concatenated to obtain concatenated data, and the concatenated data is written to the storage block allocated to the temporary dump file. Then the electronic device is powered off.

6. The method according to claim 5, characterized in that, The method further includes: When the electronic device is detected to be powered on, check whether the temporary dump file exists in the NAND Flash medium. If it exists, for the data belonging to the same file in the temporary dump file, create a cache area in the local memory corresponding to the file to which the data belonging to the same file belongs, and write the data belonging to the same file to the created cache area. After writing the data in the temporary dump file to the corresponding cache area, the data in the storage block allocated to the temporary dump file is deleted.

7. A data processing apparatus, characterized in that, The device is used in an electronic device, and the device includes: A receiving module is used to receive data write requests; the data write request indicates the identifier of the target file; The checking module is used to check whether a target cache area of ​​a specified size corresponding to the target file exists locally based on the identifier of the target file. If not, the target cache area corresponding to the target file is created in local memory. The specified size is the size of a storage page in the NAND Flash medium of the electronic device. A first processing module is configured to, if the available storage space in the target cache corresponding to the target file is less than the size of the target data to be written indicated by the data write request, truncate the target data to obtain a first part of data and a second part of data before writing the target data to the NAND Flash medium; write the first part of data to the available storage space in the target cache, the size of the first part of data matching the size of the available storage space in the target cache; then write the data in the target cache to the NAND Flash medium and clear the data in the target cache; write the second part of data to the available storage space in the target cache based on the size of the available storage space in the target cache; and when the size of the available storage space in the target cache is a preset value, write the data in the target cache to the NAND Flash medium and clear the data in the target cache. The second processing module is configured to, if the size of the available storage space in the target cache corresponding to the target file is greater than or equal to the size of the target data, write the target data into the available storage space in the target cache before writing the target data into the NAND Flash medium, and when the size of the available storage space in the target cache is a preset value, write the data in the target cache into the NAND Flash medium and clear the data in the target cache.

8. The apparatus according to claim 7, characterized in that, The apparatus further includes: a reading module, configured to receive a data reading request, the data reading request indicating an identifier of a target file; read the data to be read indicated by the data reading request from the NAND Flash medium; and, if the size of the data read from the NAND Flash medium is less than the size of the data to be read, check whether the data to be read exists in a target cache corresponding to the target file; if so, read the remaining unread data from the data to be read from the target cache, and concatenate the data read from the NAND Flash medium and the data read from the target cache to obtain the data to be read; and / or, The apparatus further includes: a determining module, configured to, after writing data from the target cache to the NAND Flash medium, determine whether the target file needs to undergo a rotation operation; if so, modify the identifier indicating the storage location of the target file already stored in the NAND Flash medium from a block identifier of a first storage block to a block identifier of a second storage block, and then erase the data in the first storage block; wherein, the storage block is a storage block in the NAND Flash medium; the storage blocks allocated to the target file include the first storage block and the second storage block; the storage time of the data in the first storage block is the longest among the storage blocks allocated to the target file, and the storage time of the data in the second storage block is closest to the storage time of the data in the first storage block; and / or, Determining whether the target file needs to undergo a rotation operation includes: checking whether the available storage space in the allocated storage block of the target file is a preset value; if yes, then determining that the target file needs to undergo a rotation operation; if no, then determining that the target file does not need to undergo a rotation operation; and / or, The device further includes: a shutdown module, configured to receive a shutdown command; if at least one buffer exists, create a temporary dump file and allocate storage blocks from the NAND Flash medium for the temporary dump file based on the size of the data in each buffer; concatenate the data in each buffer to obtain concatenated data, and write the concatenated data to the allocated storage block of the temporary dump file, and then shut down the electronic device; and / or, The device further includes: a power-on module, configured to, when the electronic device is detected to be powered on, check whether the temporary dump file exists in the NAND Flash medium; if it exists, create a cache area in local memory corresponding to the file to which the data belonging to the same file belongs for the data in the temporary dump file, and write the data belonging to the same file to the created cache area; after writing the data in the temporary dump file to the corresponding cache area, delete the data in the storage block to which the temporary dump file is allocated.

9. An electronic device, characterized in that, The electronic device includes: Processor; and A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, cause the processor to perform the steps of the method as described in any one of claims 1 to 6.

Images