A data storage synchronization method and apparatus

By configuring a caching strategy in the network file transfer protocol, data blocks are first written to the associated target file in the cache, and then stored in the long-term storage medium when the file caching conditions are met. This solves the risk of data loss and performance problems, and achieves efficient synchronous data writing.

CN114676103BActive Publication Date: 2025-10-31MACROSAN TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210210900.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-04
Publication Date
2025-10-31
Estimated Expiration
2042-03-04

AI Technical Summary

Technical Problem

In existing network file transfer protocols, asynchronous data writing poses a risk of data loss, and frequent data block synchronization operations negatively impact data writing performance.

Method used

By using a configuration file caching strategy, data blocks are first written to their associated target files in the local cache. When the file caching strategy is in effect, the target files are stored in the persistent storage medium as files, thus avoiding frequent data block synchronization operations.

Benefits of technology

While ensuring data security and consistency, the performance of synchronous data writing has been improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114676103B_ABST
    Figure CN114676103B_ABST
Patent Text Reader

Abstract

This application provides a data storage synchronization method and apparatus. When writing a target data block, the method configures a file caching strategy. If a first parameter value is not set to indicate that the data block will be written to a persistent storage medium, the target data block is written to a target file already created in the local cache and associated with the target data block. If the file caching parameter is set to indicate that the file caching strategy is effective, and it is determined based on the target data block that the target file needs to be stored in the persistent storage medium, then the target file is stored in the persistent storage medium. In this way, data block synchronization is performed on a file-by-file basis only when it is determined that the target file needs to be stored in the persistent storage medium. This ensures data security and consistency while avoiding frequent data synchronization operations on data blocks, thereby improving the performance of data synchronization writing while maintaining a certain level of data security.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of storage, and in particular to a data storage synchronization method and apparatus. Background Technology

[0002] NAS (Network Attached Storage) devices use standard network protocols to transmit data, providing file sharing and data backup services for computers with different operating systems on a network. Driven by the rise of cloud computing and big data, file transfer and sharing have become essential requirements; therefore, NAS needs an efficient, secure, and stable network file transfer protocol.

[0003] In practical applications, when using network file transfer protocols to write data to the cache, these protocols include at least CIFS (Common Internet File System) and NFS (Network File System). Asynchronous data writing operations are generally used, that is, data blocks are first written to the cache and then stored in the permanent storage medium. However, during the caching process, if the NAS device loses power, there is a risk that data blocks already in the cache may be lost because they are not saved to the permanent storage medium in time.

[0004] Given the above, to ensure data security, these network file transfer protocols synchronize the files to be cached to persistent storage. This means that data blocks are synchronously stored and written to both the cache and persistent storage. This synchronization ensures data security and consistency. However, since each data block needs to be written to persistent storage, this can impact write performance, resulting in poor performance for synchronous data writing. Therefore, while existing technologies can guarantee data security, they suffer from poor performance for synchronous data writing. Summary of the Invention

[0005] In view of this, this application provides a data storage synchronization method and apparatus, which can improve data security and data synchronization writing performance.

[0006] Specifically, this application is implemented through the following technical solution:

[0007] In a first aspect, embodiments of this application provide a data storage synchronization method applied to an electronic device, the electronic device including a cache and a persistent storage medium, the method comprising:

[0008] For each target data block to be written, read the locally configured storage parameters. If the storage parameters are not the first parameter value, where the first parameter value indicates that the data block is to be written to the persistent storage medium, then write the target data block to the target file associated with the target data block that has been created in the local cache.

[0009] After the target data block is written to the file, the locally configured file cache parameters are identified. When the file cache parameter is a second parameter value, the second parameter value indicates that the file cache policy is effective. Then, when it is determined based on the target data block that the target file needs to be stored in the long-term storage medium, the target file is stored in the long-term storage medium.

[0010] Secondly, embodiments of this application provide a data storage synchronization device applied to an electronic device, the electronic device including a cache and a persistent storage medium, the device comprising:

[0011] The cache writing unit is used to read the locally configured storage parameters for each target data block to be written. If the storage parameter is not a first parameter value, where the first parameter value is used to indicate that the data block is written to the persistent storage medium, then the target data block is written to the target file associated with the target data block that has been created in the local cache.

[0012] The parameter identification unit is used to identify locally configured file cache parameters after the target data block is written to the file. When the file cache parameter is a second parameter value, the second parameter value is used to indicate that the file cache policy is effective, and then the file writing unit is triggered.

[0013] The file writing unit is used to store the target file in the long-term storage medium when it is determined, based on the target data block, that the target file needs to be stored in the long-term storage medium.

[0014] In this application's technical solution, when an electronic device writes a target data block, a file caching strategy is pre-configured. If the first parameter value is not set to indicate that the data block is to be written to a persistent storage medium, the target data block is written to a target file already created in the local cache and associated with the target data block. If the file caching parameter is set to indicate that the preset file caching strategy is in effect, and it is determined based on the target data block that the target file needs to be stored in the persistent storage medium, then the target file is stored in the persistent storage medium. Thus, unlike existing technologies that frequently perform data synchronization operations on each written data block, this embodiment only performs data synchronization operations on a file-by-file basis when it is determined that the target file needs to be stored in the persistent storage medium. This ensures data security and consistency while avoiding frequent data synchronization operations on data blocks, thereby improving both data security and the performance of data synchronization writing. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a data storage synchronization method according to an embodiment of this application;

[0016] Figure 2 This is a flowchart illustrating the second data storage synchronization method according to an embodiment of this application;

[0017] Figure 3 This is a schematic diagram of the method for determining the target file to be stored in a long-term storage medium based on the target data block in step 103 of this application embodiment;

[0018] Figure 4 This is a schematic diagram of the structure of an electronic device shown in this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, and to make the above-mentioned objectives, features and advantages of the embodiments of the present invention more apparent and understandable, the prior art solutions and the technical solutions in the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0020] In practical applications, when using network file transfer protocols to write data to storage media, the commonly used network file transfer protocols will directly write the data to the persistent storage medium to ensure data security and consistency during data synchronization. However, when writing data, it is done in units of data blocks, and each data block must be written to the persistent storage medium. This will affect the performance of data writing to some extent, resulting in poor data writing performance.

[0021] In related technologies, network file transfer protocols rely on file systems to write data to storage media. Taking ZFS as an example, ZFS has a ZFS sync mechanism, which treats each data block as a synchronization object. The data block size is specified by the `blocksize` parameter, which sets the minimum storage unit size, defaulting to 128KB. When the `sync` parameter is set to the `always` parameter, indicating that each data block will undergo a cached synchronization operation, each data block is directly written to permanent storage media to ensure that data is not lost. Although ZFS can modify the data block size by changing the `blocksize` parameter, thereby modifying the amount of data in each synchronization operation, only one data block size can be specified for a file system. However, in actual business scenarios, the size of each file is inconsistent, leading to situations where one file undergoes multiple sync operations, and different files undergo different numbers of sync operations. Therefore, when the ZFS file system's `sync` parameter is set to `always`, the upper-layer network transmission protocol will synchronize each data block regardless of whether caching is enabled, which will still affect data write performance to some extent.

[0022] In another related technology, taking CIFS as an example of a network file transfer protocol, CIFS has a CIFS syncalways mechanism. That is, CIFS has a sync always parameter. When sync always is set to yes, regardless of whether the underlying file system's sync parameter is always, the underlying file system's fsync function will be called through the virtual file system interface to perform data synchronization. It can be seen that this technical solution also performs data synchronization operations on a data block basis, and it also cannot take into account both data security and data write performance.

[0023] To address the aforementioned technical problems, this application provides a data storage synchronization method. This method is applied to an electronic device, which includes a cache and a persistent storage medium. For each target data block to be written, locally configured storage parameters are read. If the storage parameter is not a first parameter value (where the first parameter value indicates that the data block is to be written to the persistent storage medium), the target data block is written to a target file associated with the target data block already created in the local cache. After writing the target data block to the file, locally configured file cache parameters are identified. When the file cache parameter is a second parameter value (where the second parameter value indicates that a file caching policy is in effect), if it is determined based on the target data block that the target file needs to be stored in the persistent storage medium, then the target file is stored in the persistent storage medium. As can be seen, the technical solution provided in this application does not perform frequent data synchronization operations on each written data block as in the prior art. Instead, it performs data synchronization operations on the file as a unit only when it is determined that the target file needs to be stored in the long-term storage medium. This can ensure data security and consistency, and avoid frequent data synchronization operations on data blocks. In this way, it can improve the performance of data synchronization writing while ensuring a certain level of data security.

[0024] See Figure 1 This is a flowchart of a first data storage synchronization method provided in an embodiment of this application, applied to an electronic device, the electronic device including a cache and a persistent storage medium, the method including the following steps:

[0025] Step 101: For each target data block to be written, read the locally configured storage parameters. If the storage parameters are not the first parameter value, then proceed to step 102.

[0026] The target data block can be the data block to be stored. As an example, before step 101, the method further includes: parsing the received target data packet and obtaining at least one target data block from the parsed data packet.

[0027] The storage parameters described above in this embodiment can be pre-placed in a configuration file. These storage parameters indicate whether the data block is directly written to the permanent storage medium. When the storage parameter is set to a first parameter value indicating "yes" to write the data block to the permanent storage medium, it means that the electronic device will not cache the data block when writing it; that is, it will not store it in the cache used as temporary storage, but will store it directly in the permanent storage medium. As an example, when the storage parameter is the first parameter value, the target data block is written to the cache and the permanent storage medium. When the first parameter is set to a "no" parameter value, that is, when the electronic device writes the data block, step 102 is executed.

[0028] Step 102: Write the target data block into the target file associated with the target data block that has been created in the local cache. After writing the target data block into the file, identify the locally configured file cache parameters. When the file cache parameter is the second parameter value, then execute step 103.

[0029] In this step, writing the target data block to the target file on the cache allows the target data block to be temporarily stored in the cache.

[0030] The target data block and the target file are associated. When writing data blocks to the source file to which the target data block belongs, a target file and its file handle associated with the source file are first created in the cache so that the data blocks in the source file can be cached in the target file associated with that data block. The target file can be specified using a file handle. In this embodiment, the file handle can be an alias used to record the opened target file. The file handle can be used to perform read and write operations on the file, such as if the file handle is a file descriptor (fd).

[0031] As an example, the file caching strategy of this embodiment can be to enable synchronous storage of data on a file-by-file basis. The file caching parameter is a switch set for the file caching strategy. That is, the file caching parameter is a parameter specifically configured to determine whether the preset file caching strategy is effective. When the file caching parameter is set to the second parameter used to indicate that the file caching strategy is effective, step 103 is executed. When the file caching parameter is set to the parameter value for the caching strategy to be effective, as an example, the target data block is written to the target file on the cache. After the target file is larger than the system-set threshold, the target file is synchronized to the long-term storage medium.

[0032] Step 103: When it is determined based on the target data block that the target file needs to be stored in the long-term storage medium, the target file is stored in the long-term storage medium.

[0033] In this step, when it is determined that the target data block meets the conditions for storing the target file in a persistent storage medium, the target file is stored in the persistent storage medium.

[0034] based on Figure 1 As shown in the process, in step 102, the target data block is temporarily stored in a cache as a temporary storage medium. Afterward, the target file is stored in a permanent storage medium as a file, thereby enabling data synchronization.

[0035] Figure 1 The process shown has been completed.

[0036] Therefore, the technical solution provided in this application pre-configures a file caching strategy when writing each target data block. If the first parameter value is not set to indicate that the data block is to be written to a persistent storage medium, the target data block is written to a target file already created in the local cache and associated with the target data block. If the file caching parameter is set to indicate that the preset file caching strategy is in effect, and it is determined based on the target data block that the target file needs to be stored in the persistent storage medium, then the target file is stored in the persistent storage medium. Thus, unlike the prior art, this embodiment does not frequently perform data synchronization operations on each written data block. Instead, it performs data synchronization operations on a file-by-file basis only when it is determined that the target file needs to be stored in the persistent storage medium. This ensures data security and consistency while avoiding frequent data synchronization operations on data blocks, thereby improving both data security and the performance of data synchronization writing.

[0037] exist Figure 1 After the illustrated process is completed, as an example, in step 103, it is determined that the target file needs to be stored in the persistent storage medium based on the target data block, including:

[0038] Determine whether the target data block is the last data block in the source file to which it belongs. If so, determine that the target file needs to be stored in the long-term storage medium.

[0039] In this embodiment, data blocks are propagated in the form of data packets in the network transmission protocol. Based on this, after the target data packet is parsed, it also includes a target identifier corresponding to the target data block, so as to determine whether the target data block is the last data block in the source file to which the target data block belongs. As an example, the target identifier corresponding to the target data block is read from the target data packet to which the target data block belongs, and it is determined whether the target identifier is an identifier that indicates that the target data block is the last data block in the source file to which the target data block belongs.

[0040] The source file to which the target data block belongs is divided into two types: an existing file and a newly created file. As one example, when the source file is an existing file, a target file is created when it is determined that a data block will be written to it. The target identifier is the sequential number of the data block written to the target file. When the target identifier represents the last data block written to the target file from the source file, it indicates that the target data block corresponding to that target identifier is the last data block in the source file. As another example, when the source file is a newly created file, a target file is created when it is determined that a data block will be written to it. When the target identifier represents an operation identifier indicating that the source file is closed, it indicates that the target data block corresponding to that target identifier is the last data block in the source file.

[0041] As another embodiment, if the target data block is not the last data block in the source file to which the target data block belongs, then the next target data block to be written is taken as the new target data block, and steps 101 to 103 are repeated.

[0042] As an example, after step 103, when the file cache parameter is a parameter value used to indicate that the file cache policy is effective, the next target data block is used as the new target data block, and steps 101 to 103 are repeated.

[0043] As another embodiment, such as Figure 2 As shown, in step 103, determining whether the target file needs to be stored in the persistent storage medium based on the target data block includes:

[0044] Step 1031: Determine whether the size of the target file is greater than or equal to the set file cache threshold. If yes, proceed to step 1032; otherwise, proceed to step 1033.

[0045] In this embodiment, the file caching threshold is a pre-set threshold. The file caching threshold can be obtained based on actual experience, such as the failure frequency of this electronic device within a specified time period, or based on the business type of the data to be stored. For data with high data security requirements, the file caching threshold can be set to a larger value, and vice versa.

[0046] Step 1032: Determine that the target file needs to be stored in the long-term storage medium.

[0047] Step 1033: Determine whether the target data block is the last data block in the source file to which the target data block belongs. If the target data block is the last data block in the source file to which the target data block belongs, determine that the target file needs to be stored in the long-term storage medium.

[0048] As can be seen, in this embodiment, if the size of the data block stored in the target file is greater than or equal to the aforementioned file caching threshold, it means that the temporarily stored data block in the target file is large enough. For data security and to prevent data loss due to electronic device failure before the data block is fully cached, a larger target file can be synchronized in the long-term storage medium. This further ensures compatibility between data synchronization performance and data security. If the size of the temporarily stored data block in the target file has not yet reached the aforementioned file caching threshold, it means that the aforementioned data block can still be cached in the target file. That is, the next target data block can be temporarily cached so that the last target data block of the target file is cached. Alternatively, when the target file reaches the file caching threshold, the target file is stored in the long-term storage medium as a file.

[0049] To facilitate understanding, a detailed description of a specific embodiment is provided below. Taking the CIFS network file transfer protocol as an example, the technical solution of this application is illustrated as follows: A file caching parameter, namely the global parameter "whole file sync," is added to the existing smb.conf configuration file of CIFS. This parameter serves as a switch to enable or disable the file caching policy. The "whole file sync" parameter is either set by the user to enable the file caching policy or set to disable it. To enable synchronized storage of data blocks at the file level, the "whole file sync" parameter is set to enable the file caching policy.

[0050] When writing a data block begins, it is determined whether the specified parameter, 'sync', is set to the value indicating that the data block will be written to the persistent storage medium. Specifically, it is checked whether the 'sync' parameter is 'always'. If it is not set to 'always', it means the data block is first cached in the temporary storage medium. Based on this, the data block to be stored is written to the target file in the cache. The 'whole file sync' parameter is retrieved from the configuration file, and it is determined whether the 'whole file sync' parameter indicates that the preset file caching strategy is in effect. If so, and with the 'whole file sync' parameter enabled, it is further determined whether the size of the target file is greater than or equal to the set file caching threshold. If so, it is determined that the target file needs to be stored in the persistent storage medium. If not, it is determined whether the data block is the last data block in the source file to which it belongs. If the data block is the last data block in the target file, then the persistent storage medium synchronization operation is performed on the target file.

[0051] Corresponding to the aforementioned data synchronization method embodiments, this application also provides an embodiment of a data storage synchronization device.

[0052] See Figure 3 Here is a block diagram illustrating an embodiment of a data storage synchronization device 300 shown in this application: Figure 3 As shown, this device is applied to an electronic device, which includes a cache and persistent storage medium. The device includes:

[0053] The cache writing unit 301 is used to read the locally configured storage parameters for each target data block to be written. If the storage parameter is not a first parameter value, where the first parameter value is used to indicate that the data block is written to the long-term storage medium, then the target data block is written to the target file associated with the target data block that has been created in the local cache.

[0054] The parameter identification unit 302 is used to identify locally configured file cache parameters after the target data block is written to the file. When the file cache parameter is a second parameter value, the second parameter value is used to indicate that the file cache policy is effective, and then the file writing unit is triggered.

[0055] The file writing unit 303 is used to store the target file in the long-term storage medium when it is determined, based on the target data block, that the target file needs to be stored in the long-term storage medium.

[0056] As one embodiment, the file writing unit includes:

[0057] The first determination subunit is used to determine whether the target data block is the last data block in the source file to which the target data block belongs. If so, it determines that the target file needs to be stored in the persistent storage medium; or...

[0058] The second determination subunit is used to determine whether the size of the target file is greater than or equal to a set file cache threshold. If yes, it determines that the target file needs to be stored in the long-term storage medium. If no, it determines whether the target data block is the last data block in the source file to which the target data block belongs. If the target data block is the last data block in the source file to which the target data block belongs, it determines that the target file needs to be stored in the long-term storage medium.

[0059] As one embodiment, the first determination subunit is specifically used for:

[0060] Read the target identifier corresponding to the target data block from the target data packet to which the target data block belongs, and determine whether the target identifier is an identifier that indicates that the target data block is the last data block in the source file to which the target data block belongs.

[0061] As one embodiment, the device further includes:

[0062] A deletion unit is configured to delete the target file from the cache when a deletion condition for deleting the target file from the cache is met, wherein the deletion condition includes at least a time threshold and / or the cache size threshold.

[0063] As an example, if the storage parameter is a first parameter value, the device further includes: a persistent storage unit for writing the target data block into the cache and the persistent storage medium.

[0064] Therefore, when writing target data blocks, the electronic device pre-configures a file caching strategy. If the first parameter value is not set to indicate that the data block is to be written to a persistent storage medium, the target data block is written to a target file already created in the local cache and associated with the target data block. If the file caching parameter is set to indicate that the preset file caching strategy is in effect, and it is determined based on the target data block that the target file needs to be stored in the persistent storage medium, then the target file is stored in the persistent storage medium. Thus, unlike existing technologies that frequently perform data synchronization operations on each written data block, this embodiment only performs data synchronization operations on a file-by-file basis when it is determined that the target file needs to be stored in the persistent storage medium. This ensures data security and consistency while avoiding frequent data synchronization operations on data blocks, thereby improving both data security and the performance of data synchronization writing.

[0065] The embodiments of the data storage synchronization device of this application can be applied to electronic devices. The device embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of the electronic device loading the corresponding computer program instructions from non-volatile memory into memory for execution.

[0066] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0067] The electronic device provided in this application, from a hardware perspective, can be found in the hardware architecture diagram. Figure 4 As shown, it includes: a machine-readable storage medium and a processor, wherein: the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the data storage synchronization operation disclosed in the above example.

[0068] The machine-readable storage medium provided in this application embodiment stores machine-executable instructions. When the machine-executable instructions are invoked and executed by a processor, the machine-executable instructions cause the processor to implement the data storage synchronization operation disclosed in the above examples.

[0069] Here, a machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, a machine-readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0070] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0071] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0072] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0074] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0075] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0076] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0077] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data storage synchronization method, characterized in that, Applied to an electronic device, the electronic device including a cache and persistent storage medium, the method includes: For each target data block to be written, read the locally configured storage parameters. If the storage parameters are not the first parameter value, where the first parameter value indicates that the data block is to be written to the persistent storage medium, then write the target data block to the target file associated with the target data block that has been created in the local cache. After writing the target data block to the file, the locally configured file cache parameters are identified. When the file cache parameter is a second parameter value, the second parameter value indicates that the file cache policy is in effect. Then, when it is determined based on the target data block that the target file needs to be stored in the persistent storage medium, the target file is stored in the persistent storage medium. Determining that the target file needs to be stored in the persistent storage medium based on the target data block includes: determining whether the target data block is the last data block in the source file to which the target data block belongs; if so, determining that the target file needs to be stored in the persistent storage medium; or, determining whether the size of the target file is greater than or equal to a set file cache threshold; if so, determining that the target file needs to be stored in the persistent storage medium; if not, determining whether the target data block is the last data block in the source file to which the target data block belongs; if the target data block is the last data block in the source file to which the target data block belongs, determining that the target file needs to be stored in the persistent storage medium.

2. The method according to claim 1, characterized in that, The step of determining whether the target data block is the last data block in the source file to which the target data block belongs includes: Read the target identifier corresponding to the target data block from the target data packet to which the target data block belongs, and determine whether the target identifier is an identifier that indicates that the target data block is the last data block in the source file to which the target data block belongs.

3. The method according to claim 1, characterized in that, After determining that the target file needs to be stored in the persistent storage medium based on the target data block, the method further includes: When the deletion conditions for deleting the target file from the cache are met, the target file is deleted from the cache, and the deletion conditions include at least a time threshold and / or a cache size threshold.

4. The method according to claim 1, characterized in that, If the storage parameter is the first parameter value, the method further includes: writing the target data block into the cache and the persistent storage medium.

5. A data storage synchronization device, characterized in that, Applied to an electronic device, the electronic device including a cache and persistent storage medium, the device includes: The cache writing unit is used to read the locally configured storage parameters for each target data block to be written. If the storage parameter is not a first parameter value, where the first parameter value is used to indicate that the data block is written to the persistent storage medium, then the target data block is written to the target file associated with the target data block that has been created in the local cache. The parameter identification unit is used to identify locally configured file cache parameters after the target data block is written to the file. When the file cache parameter is a second parameter value, the second parameter value is used to indicate that the file cache policy is effective, and then the file writing unit is triggered. A file writing unit is configured to store the target file in the long-term storage medium when it is determined, based on the target data block, that the target file needs to be stored in the long-term storage medium. The file writing unit includes: a first determining subunit, configured to determine whether the target data block is the last data block in the source file to which the target data block belongs; if so, determining that the target file needs to be stored in the long-term storage medium; or, a second determining subunit, configured to determine whether the size of the target file is greater than or equal to a set file cache threshold; if so, determining that the target file needs to be stored in the long-term storage medium; if not, determining whether the target data block is the last data block in the source file to which the target data block belongs; if the target data block is the last data block in the source file to which the target data block belongs, determining that the target file needs to be stored in the long-term storage medium.

6. The apparatus according to claim 5, characterized in that, The first judgment subunit is specifically used for: Read the target identifier corresponding to the target data block from the target data packet to which the target data block belongs, and determine whether the target identifier is an identifier that indicates that the target data block is the last data block in the source file to which the target data block belongs.

7. The apparatus according to claim 5, characterized in that, The device also includes: A deletion unit is configured to delete the target file from the cache when a deletion condition for deleting the target file from the cache is met, wherein the deletion condition includes at least a time threshold and / or a cache size threshold.

8. The apparatus according to claim 5, characterized in that, If the storage parameter is a first parameter value, the device further includes a persistent storage unit for writing the target data block into the cache and the persistent storage medium.

Citation Information

Patent Citations

  • ZFS file system-based data storage method and apparatus

    CN108363727A

  • NAS cluster metadata processing method and device, NAS gateway and medium

    CN111399760A