File management method based on raid card, storage medium and computer device

Abstract

The application discloses a file management method based on a RAID card, a storage medium and computer equipment. The method comprises the following steps: creating a lower RAID volume and an upper RAID volume of an Overlay volume based on a RAID card; writing basic common files into the lower RAID volume, wherein the data in the basic common files is associated with corresponding logical block addresses; and storing an initial mapping table in the upper RAID volume, wherein the upper RAID volume is also used for storing modified data of the basic common files, the mapping table is used for mapping the logical block addresses associated with the data in the Overlay volume, and the modification state of each logical block address is recorded. Through the above method, the application can realize the logical block address level Overlay based on the RAID hardware, is favorable for reducing the space waste of the existing software implemented file system, enables the newly created Overlay volume to quickly locate and access data, improves the read-write performance of the Overlay, can be compatible with various file systems, and has wide applicability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a file management method based on RAID card, storage medium and computer equipment. BACKGROUND

[0002] In the traditional file system, the modification of file data usually requires copying and rewriting the entire file or data block, which not only consumes valuable storage space, but also reduces the efficiency of file processing. To solve this problem, Overlay file system in container technology emerges as the times require. Overlay file system divides data storage into multiple layers, such as Lower layer, Upper layer and Merge layer, to realize hierarchical storage and management of data. Among them, Lower layer usually contains read-only basic data such as operating system image, fixed part of application program, etc., Upper layer is used to store read-write modified and new data such as container configuration file, log, etc., and Merge layer is used to merge the directory information of Lower layer and Upper layer to provide the file system view required by container runtime. This hierarchical storage method not only improves the flexibility of file data modification, but also optimizes the performance of data reading, so that the container can be quickly started, the coexistence of multiple versions of software in the same system is realized, and the efficiency of file management is improved.

[0003] However, since the existing software Overlay system mainly operates at the file level, it means that when the file needs to be modified, even if only part of the content in the file needs to be updated, the entire file needs to be copied to the Upper layer, which undoubtedly causes waste of space, and also consumes additional time and computing resources in the copying process. The read-write operation is relatively complex, for example, the write operation involves multiple layer operations, resulting in additional overhead, and the read operation also causes performance loss due to the need to search multiple layers, and the read-write efficiency is low. At the same time, since the granularity of the software-based Overlay file system is at the file level, it depends on specific operating systems and kernel versions, so in some old operating systems or kernels that do not support Overlay file system functions, some applications may not adapt to the hierarchical architecture and features of Overlay file system and cannot work normally, and the compatibility is poor. SUMMARY

[0004] The present application mainly provides a file management method based on RAID card, storage medium and computer equipment, aiming at solving the technical problem of low read-write efficiency of the existing Overlay file system.

[0005] To solve the above technical problems, the technical scheme adopted by the present application is to provide a file management method based on RAID card. The file management method based on RAID card comprises:

[0006] creating, by a RAID card, an under-layer RAID volume and an upper-layer RAID volume for a new Overlay volume; writing, in the under-layer RAID volume, a base common file, data in the base common file being associated with a corresponding logical block address; storing, in the upper-layer RAID volume, an initial mapping table, wherein the upper-layer RAID volume is further used to store modified data of the base common file, and the mapping table is used to map the logical block address associated with data in the Overlay volume and record a modification state of each logical block address.

[0007] In some embodiments, the file management method based on the RAID card further comprises:

[0008] in response to receiving the write instruction, confirming the logical block address and the write data corresponding to the write instruction; storing the write data into the upper-layer RAID volume and updating the modification state corresponding to the logical block address in the mapping table.

[0009] In some embodiments, the file management method based on the RAID card further comprises:

[0010] in response to receiving the read instruction, confirming the logical block address corresponding to the read instruction; based on the mapping table, determining data of the logical block address from the upper-layer RAID volume and / or the under-layer RAID volume.

[0011] In some embodiments, based on the mapping table, determining data of the logical block address from the upper-layer RAID volume and / or the under-layer RAID volume comprises: based on the mapping table, determining that the modification state corresponding to the logical block address is an initial state, reading data of the logical block address from the under-layer RAID volume to a temporary data storage area; based on the mapping table, determining that the modification state corresponding to the logical block address is a post-write state, reading data of the logical block address from the upper-layer RAID volume to the temporary data storage area.

[0012] In some embodiments, the file management method based on the RAID card further comprises:

[0013] determining that the mapping table entry corresponding to the logical block address is in the upper-layer RAID volume, reading the mapping table entry from the mapping table of the upper-layer RAID volume to a dynamic random access memory; based on the mapping table entry in the dynamic random access memory, performing file read / write.

[0014] In some embodiments, the RAID card-based file management method further comprises: determining that the mapping table entry in the dynamic random access memory is different from the corresponding mapping table entry in the upper RAID volume, and marking the mapping table entry in the dynamic random access memory as dirty data; determining that the data size of the dirty data in the dynamic random access memory reaches a preset dirty data size threshold, and that no target logical block address is queried from each mapping table entry of the dynamic random access memory, reading the corresponding mapping table entry from the upper RAID volume, and storing the corresponding mapping table entry in the dynamic random access memory.

[0015] In some embodiments, after determining that the data size of the dirty data in the dynamic random access memory reaches a preset dirty data size threshold, and that no target logical block address is queried from each mapping table entry of the dynamic random access memory, reading the corresponding mapping table entry from the upper RAID volume, and storing the corresponding mapping table entry in the dynamic random access memory, the method further comprises: determining the least recently used degree of each mapping table entry in the dynamic random access memory based on a least recently used algorithm; and storing the mapping table entries of a preset persistence size in the dynamic random access memory in the upper RAID volume in descending order of the least recently used degree, the preset persistence size being equal to the size of the mapping table entry read from the upper RAID volume.

[0016] In some embodiments, the preset dirty data size threshold is the smaller of the storage size available for storing the mapping table entry in the dynamic random access memory and the storage size that can be safely persisted when the dynamic random access memory is powered off.

[0017] To solve the above technical problems, another technical solution adopted by the present application is to provide a storage medium, the storage medium storing program data, the program data being executed by a processor to implement the steps of the RAID card-based file management method as described above.

[0018] To solve the above technical problems, another technical solution adopted by the present application is to provide a computer device, the computer device comprising a processor and a memory connected to each other, the memory storing a computer program, and the processor executing the computer program to implement the steps of the RAID card-based file management method as described above.

[0019] The beneficial effects of the present application are: different from the prior art, the present application discloses a file management method based on a RAID card, a storage medium and a computer device. The present application creates a lower RAID volume and an upper RAID volume based on a new Overlay volume of the RAID card; writes basic common files in the lower RAID volume, and the data in the basic common files is associated with corresponding logical block addresses; stores an initial mapping table in the upper RAID volume, wherein the upper RAID volume is also used to store modified data of the basic common files, the mapping table is used to map the logical block addresses associated with the data in the Overlay volume, and record the modification state of each logical block address. The logical block address level Overlay based on the RAID hardware is realized, so that the files stored in the Overlay volume can be managed with high granularity, not only reducing unnecessary data replication and improving read-write efficiency, avoiding the space waste of the existing software implemented file system, but also because of the existence of the mapping table, the new Overlay volume can quickly locate and access data, improve the read-write performance of the Overlay volume, and is beneficial to optimize the overall performance of the host device. At the same time, the file management method does not need to depend on specific operating systems and kernel versions, has good compatibility and flexibility, can adapt to file management requirements in different environments, and has wide applicability. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings, wherein:

[0021] Figure 1 is a flowchart of an embodiment of the file management method based on the RAID card provided by the present application;

[0022] Figure 2 is a flowchart of another embodiment of the file management method based on the RAID card provided by the present application;

[0023] Figure 3 is a flowchart of another embodiment of the file management method based on the RAID card provided by the present application;

[0024] Figure 4 is Figure 1 is a flowchart of an embodiment of step 32 in the embodiment;

[0025] Figure 5 is a flowchart of another embodiment of the file management method based on the RAID card provided by the present application;

[0026] Figure 6 is a flowchart of another embodiment of the RAID card-based file management method provided in the present application;

[0027] Figure 7 is Figure 6 is a flowchart of an embodiment after step 52 in the embodiment;

[0028] Figure 8 is a structural diagram of an embodiment of the storage medium provided in the present application;

[0029] Figure 9 is a structural diagram of an embodiment of the computer device provided in the present application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be clearly and completely described in the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0031] The terms “first”, “second”, “third” in the embodiments of the present application are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by “first”, “second”, “third” can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of “a plurality of” is at least two, for example, two, three, etc., unless otherwise specifically limited. In addition, the terms “include” and “have” and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0032] In this document, the term “embodiment” means that the specific features, structures or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The phrase appears in various places in the specification does not necessarily refer to the same embodiment, nor is it mutually exclusive or alternative to other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0033] The present application provides a RAID card-based file management method, referring to Figure 1 , Figure 1This is a flowchart illustrating an embodiment of a file management method based on a RAID card provided in this application. The file management method based on a RAID card includes:

[0034] Step 11: Create a lower-level RAID volume and an upper-level RAID volume based on the RAID card for the overlay volume.

[0035] Step 12: Write the underlying common file to the lower-level RAID volume. The data in the underlying common file is associated with the corresponding logical block address.

[0036] Step 13: Store the initial mapping table in the upper-layer RAID volume. The upper-layer RAID volume is also used to store the modified data of the underlying public files. The mapping table is used to map the logical block addresses associated with the data in the Overlay volume and record the modification status of each logical block address.

[0037] In this embodiment, RAID (Redundant Array of...) Independent Disks (RAID) volumes are a technology that combines multiple independent physical storage disks into a single logical storage unit. Through specific algorithms such as data partitioning, data mirroring, and verification mechanisms, as well as data distribution methods like RAID 0, RAID 1, RAID 5, and RAID 6, it achieves redundant data storage, providing better data storage performance and security than a single storage disk. This technology can be implemented using a RAID card installed in the host device, or by connecting or inserting an external RAID card. The RAID card is the hardware interface connecting the physical storage disks and the host. The host accesses and manages the data on the RAID volume through the RAID card. RAID provides the functionality to manage and access multiple physical storage disks, and the file system utilizes these functions to read and write data and implement the overlay function designed in this embodiment.

[0038] In this embodiment, the RAID card can create a lower-layer RAID volume and an upper-layer RAID volume in one or more physical storage disks corresponding to the RAID card to meet the Overlay function requirements of the file system based on RAID hardware. Among them, the two RAID volumes can be configured based on different storage strategies through specific RAID levels, for example, RAID 1 can be used to realize data mirroring to ensure high reliability of data, or RAID 5 or RAID 6 can be used to balance performance and fault tolerance capability to realize redundant storage of data and performance improvement. Specifically, the lower-layer RAID volume is used to store basic common files. These basic common files are usually system-level, read-only, and not easily changed, and constitute the cornerstone of the Overlay volume, providing a stable and reliable data storage environment for upper-layer applications and users. The upper-layer RAID volume allows users to read and write to it to store write data of user file modification and newly created files. By combining the lower-layer RAID volume and the upper-layer RAID volume, a logical Overlay volume is constructed, which can provide a unified and flexible file storage space to realize efficient management of the file system through the Overlay volume.

[0039] In this embodiment, the data of the basic common files will be written into the lower-layer RAID volume, and the data of these basic common files is associated with the logical block address (LBA). Among them, the logical block address (LBA) is a general mechanism for describing the block where the data is located on the computer storage device. By mapping each storage block in the storage disk to a unique logical address, the operating system can access the storage disk data through the logical address instead of the physical location, thereby simplifying the management and data access of the storage disk. It is a logical number used to locate the data block on the storage device, usually numbered from 0 and incremented sequentially. The logical block address is an abstract concept, which does not necessarily correspond to the real physical address on the storage device, but is responsible for mapping from logical block address to physical address by the controller of the storage device. In this way, the file system can efficiently manage and access these files. When reading or writing data is needed, the file system can quickly locate the storage location of the data in the lower-layer RAID volume through its corresponding logical block address. This mapping relationship not only improves the efficiency of file access, but also provides convenience for subsequent write and read Overlay functions.

[0040] In this embodiment, the mapping table (Map Table) maps the logical block addresses associated with the data in the Overlay volume and records the modification status of the corresponding logical block addresses, i.e., which logical block addresses correspond to original data and which correspond to data written with modified or new data. The entries contained in the mapping table correspond one-to-one to each logical block address, and each entry contains a bit flag indicating whether the data corresponding to the logical block address is data after a write operation. When the data corresponding to the logical block address is modified, the flag is set to indicate the state after the write, and the mapping table is updated to ensure that the file system can quickly and accurately determine the data modification status of the corresponding logical block address. Specifically, all logical block address LBA original data flags are all 0, and when the data of an LBA is modified or updated, the LBA flag where it is located will be set to 1. The update mechanism of the mapping table ensures that the correct data location can be accurately located when performing read and write operations. In addition, the structure of the mapping table allows for quick retrieval and update, thereby improving the overall efficiency of file management. Through the use of the mapping table, the file system can effectively manage the data in the Overlay volume, achieve fine-grained control of files, and improve the processing efficiency and reliability of the file system.

[0041] In this embodiment, by storing the mapping table in the upper RAID, the persistence of the mapping table is achieved, so that even if the program exits or the system crashes, the data can still be preserved for subsequent use. Through the persistent mapping table, fast data positioning and state updating can be achieved, so that the file system can track and manage file changes in the Overlay volume in real time. In addition, the presence of the mapping table allows the file system to avoid unnecessary data copying when performing read and write operations, thereby improving the overall read and write efficiency.

[0042] The embodiment realizes a process of creating an Overlay volume through the above steps, and the obtained Overlay volume can provide an efficient, flexible and reliable file storage space. In the Overlay volume, the lower RAID volume serves as a stable base layer and stores the basic public files which cannot be easily changed, thereby providing a solid data storage cornerstone for the upper application and user, and the upper RAID volume serves as a writable layer and allows the user to perform free read and write operations, thereby storing the modified and newly created files and other data of the user and realizing the redundant storage of data and the performance improvement. Thus, the application realizes the logical block address level Overlay based on the RAID hardware, so that the file stored in the Overlay volume can be managed in a high fine-grained manner, which not only reduces unnecessary data replication, improves the read and write efficiency, avoids the space waste of the existing software implemented file system, but also enables the quick positioning and access of data based on the newly created Overlay volume due to the existence of the mapping table, thereby improving the read and write performance of the Overlay volume and being beneficial to the optimization of the overall performance of the host device. Meanwhile, the file management manner does not need to depend on the specific operating system and kernel version, has good compatibility and flexibility, can adapt to the file management requirements in different environments, and has wide applicability.

[0043] Optionally, referring to Figure 2 , Figure 2 is a flowchart of another embodiment of the file management method based on the RAID card provided by the application. The file management method based on the RAID card further includes:

[0044] Step 21: In response to receiving the write instruction, confirming the logical block address and the write data corresponding to the write instruction.

[0045] Step 22: Storing the write data into the upper RAID volume and updating the modification state corresponding to the logical block address in the mapping table.

[0046] In this embodiment, the write instruction is issued by the host system or application program, aiming to write modified or newly created data into the Overlay volume. The instruction includes the logical block address to be written and the data to be written. When the host system or application program issues the write instruction, it will first respond to the instruction and determine the logical block address corresponding to the write data. Then the write data is stored in the upper RAID volume, and the modification state of the corresponding logical block address in the mapping table is updated. The update of the mapping table ensures that the Overlay volume can track which data is modified and which is original data. In this way, the Overlay volume can effectively manage the data in the Overlay volume, ensuring the consistency and integrity of the data. In addition, the real-time update mechanism of the mapping table also ensures that the data state will not be lost in the case of system crash or program exit, thereby improving the stability and reliability of the system.

[0047] In this embodiment, the flow of this step ensures that the data in the lower RAID volume remains in a read-only state, thereby ensuring the integrity of the original base common file, while the write operation in the upper RAID volume can reflect the changes required by the host system or application program, realizing real-time update of file data. This design not only protects the base file from accidental modification, but also allows the system to quickly respond to user modification requests, improving the system's response speed and data processing flexibility. Through this way, the storage disk space utilization can be greatly improved, and the Overlay volume can be quickly started.

[0048] In this embodiment, based on the creation of the Overlay volume and the corresponding modification method, when the user needs to modify the file, the base common file can be kept unchanged. Therefore, even in a multi-user shared base common file environment, the modification operation of one user will not affect the access of other users to the original file. For example, during the operating system upgrade process, new files and modifications can be stored in a new upper RAID volume, separate from the original base operating system layer. If the upgrade fails, the lower RAID volume can be easily rolled back to the lower RAID volume, facilitating the corresponding test operation. In the scenario of multiple versions of software coexisting, this way can also realize the coexistence of multiple versions of software in the same system. Since the files of different versions of software can be stored in different upper RAID volumes and share part of the files in the lower RAID volume, it is convenient to switch between different versions of software, and the storage disk space occupation is also reduced. For example, a development environment may need to test multiple versions of a software at the same time, which can help to realize file storage and switching of different versions.

[0049] In this embodiment, after the step 21 is completed, that is, the data modification is confirmed to be successful, and the data is successfully written into the upper RAID volume, the corresponding mapping table is updated. The modification state of the logical block address in the mapping table refers to marking or modifying the entry of the logical block address corresponding to the data in the mapping table, so as to reflect that the data of the logical block address has been modified or new data has been added on the basis of the lower RAID volume, and the new version is stored to reflect the latest state of the file. The update of the mapping table includes the modification of the flag bit, because the flag bit indicates that the data of the logical block address is in the lower RAID volume in the initial state, and after the writing modification, the data is stored in the upper RAID volume, so that the correct data position can be accurately located through the flag bit. For example, when the modification state corresponding to the flag bit is the original state, the original data of the logical block address can be read from the lower RAID volume, and when the modification state corresponding to the flag bit is the state after the writing modification, the data of the logical block address after the writing modification can be read from the upper RAID volume.

[0050] In this embodiment, there are also some situations that cannot complete the step 21, such as hardware failure, permission problem or insufficient storage space, and the like. In these situations, the Overlay volume needs to have a corresponding error processing mechanism to ensure the stable operation of the system and the safety of the data, such as retrying the writing operation, recording error logs or reporting errors to the host, and the like. However, no matter what measures are taken, the logical block address related to the failed writing operation and the state of the related data in the mapping table will not be updated, so that the system can maintain the consistency and integrity of the data, avoid the risk of data damage or inconsistency, and ensure that the stability and reliability of the file management can be maintained even in the face of abnormal situations, and the safety of the user data is ensured.

[0051] Optionally, referring to Figure 3 , Figure 3 is a flowchart of another embodiment of the file management method based on the RAID card provided in the present application. The file management method based on the RAID card further includes:

[0052] Step 31: In response to receiving the reading instruction, the logical block address corresponding to the reading instruction is confirmed.

[0053] Step 32: Based on the mapping table, the data of the logical block address is determined to be read from the upper RAID volume and / or the lower RAID volume.

[0054] In this embodiment, the read instruction is issued by the host system or application to request reading the data stored in the Overlay volume at a specific logical block address, and thus the read instruction contains the logical block address to be read. When the host system or application issues the read instruction, the Overlay volume first responds to the instruction and determines the logical block address from which the data needs to be read. Then, it decides whether the data should be read from the upper RAID volume or the lower RAID volume based on the information in the mapping table. If the mapping table shows that the data at the logical block address is original, the system will directly read the data from the lower RAID volume; if the mapping table shows that the data at the logical block address is modified, the system will read the data from the upper RAID volume. In this way, the system can ensure that the read data is the latest, while maintaining the protection of the underlying common file. In addition, the presence of the mapping table enables the system to quickly locate the correct data position, thereby improving the efficiency of the read operation. In a multi-user environment, this read mechanism ensures that users can access the latest file status in real time without being affected by the operations of other users. For example, in a multi-person collaborative project, each user can see the changes made by other users without interfering with the access of other users to the file. This mapping table-based read mechanism not only improves the speed of data access, but also enhances the concurrent processing capability of the system, making the file management method more efficient and reliable.

[0055] In this embodiment, through this mapping table-based read method, the Overlay volume can efficiently handle read requests while ensuring the accuracy and speed of read operations. Due to the presence of the mapping table, the Overlay volume can quickly locate the correct data position, avoiding unnecessary data search and read delay, thereby improving the overall file access efficiency, providing an efficient, flexible and reliable file read mechanism that can meet the speed and accuracy requirements of modern file systems when handling large amounts of data. Through the use of the mapping table, the Overlay volume can quickly respond to read requests and provide the correct data version, whether it is an original or modified file. This mechanism not only improves the efficiency of file access, but also enhances the stability and reliability of the Overlay volume, providing users with a smoother and safer file usage experience.

[0056] Optionally, referring to Figure 4 , based on the mapping table, determining to read the data of the logical block address from the upper RAID volume and / or the lower RAID volume can be performed as follows:

[0057] Step 321: Based on the mapping table, determine that the modification state corresponding to the logical block address is the initial state, read the data of the logical block address from the lower RAID volume to the temporary data storage area. Step 322: Based on the mapping table, determine that the modification state corresponding to the logical block address is the post-write state, read the data of the logical block address from the upper RAID volume to the temporary data storage area.

[0058] In this optional embodiment, the temporary data storage area (Buffer) is an area for temporarily storing data, which can improve the efficiency of file reading. When the data corresponding to the logical block address needs to be read, the mapping table will be checked first to determine the modification state of the logical block address. If the mapping table shows that the data of the logical block address is original, i.e. in the unmodified state, the system will read the data from the lower RAID volume to the temporary data storage area. The use of the temporary data storage area can reduce the number of direct accesses to the lower RAID volume, thereby reducing the read pressure on the underlying common file, and also improving the read speed, because the temporary data storage area is usually located in the memory, and the access speed is faster than directly reading from the storage disk. Once the data is read into the temporary data storage area, the read request of the host system can be quickly responded, and the required data can be provided. On the contrary, if the mapping table shows that the data of the logical block address is in the modified state, the system will read the data from the upper RAID volume to the temporary data storage area. The data in the upper RAID volume reflects the latest file state, including any changes made by the user. In this way, the system can ensure that the user always accesses the latest version of the file, and is not affected by the operations of other users.

[0059] In this optional embodiment, the temporary data storage area is adapted to the size of the data corresponding to the logical block address, and is dynamically adjusted according to the size of the data to be read, so as to ensure that the entire data can be stored without overflow, and the temporary data storage area is not too large to waste hardware resources. Make the read and write mainly rely on the temporary storage data during the reading process, so as to avoid direct operation on the original storage medium, reduce the wear and tear of the storage device, and at the same time improve the efficiency of the read operation. By loading the required data into the temporary storage area and returning the buffered data in the temporary storage area to the host system or application program corresponding to the file read instruction, the waiting time can be reduced and the file access speed can be improved.

[0060] In this optional embodiment, since the target data to be read can exist in both the upper RAID volume and the lower RAID volume, the system reads data from both RAID volumes to the temporary data storage area in parallel. For example, when 4 consecutive logical block addresses LBA0-LBA3 need to be read, LBA1 is in the upper RAID volume and the rest are in the lower RAID volume, the reading process can be split into 3 parallel CMDs: 1. read LBA0 from the lower RAID volume; 2. read LBA1 from the upper RAID volume; 3. read LBA2 and LBA3 from the lower RAID volume. For this scenario, since data is read from both volumes simultaneously, it will be faster than reading from one volume, further improving the efficiency of file reading, reducing the delay of data processing, and improving the response speed and performance of the entire Overlay volume.

[0061] Optionally, referring to Figure 5 , Figure 5 is a flowchart of another embodiment of the RAID card-based file management method provided by the present application, which further comprises:

[0062] Step 41: determine that the mapping table entry corresponding to the logical block address is in the upper RAID volume, read the mapping table entry from the mapping table of the upper RAID volume to the dynamic random access memory,

[0063] Step 42: perform file reading and writing based on the mapping table entry in the dynamic random access memory.

[0064] In this embodiment, a dynamic random access memory (DRAM) is introduced to further optimize the performance of file management. The dynamic random access memory is a storage space provided by the RAID card, which uses the abnormal power-off persistence capability provided by the RAID card to protect the mapping table data from being lost when the system is powered off or fails. When the mapping table needs to be accessed, it is first checked whether the mapping table entry corresponding to the logical block address already exists in the dynamic random access memory. If the mapping table entry is already in the dynamic random access memory, the mapping table can be directly used for file reading and writing operations, which can significantly reduce the number of accesses to the storage disk with relatively slow access speed, thereby speeding up the response speed of the Overlay volume. If the mapping table entry is not in the dynamic random access memory, the corresponding mapping table entry needs to be read from the mapping table of the upper RAID volume to the dynamic random access memory first, and then file reading and writing operations are performed. In this way, the Overlay volume can utilize the high-speed access characteristics of the dynamic random access memory to reduce the number of accesses to the upper RAID volume, thereby improving the efficiency of file management.

[0065] Optionally, referring to Figure 6 , Figure 6 is a flowchart of another embodiment of the RAID card-based file management method provided in the present application, which further comprises:

[0066] Step 51: determining that the mapping table entry in the dynamic random access memory is different from the corresponding mapping table entry in the upper RAID volume, and marking the mapping table entry in the dynamic random access memory as dirty data.

[0067] Step 52: determining that the data size of the dirty data in the dynamic random access memory reaches a preset dirty data size threshold, and no target logical block address is queried from each mapping table entry of the dynamic random access memory, reading the corresponding mapping table entry from the upper RAID volume, and storing the corresponding mapping table entry into the dynamic random access memory.

[0068] In the embodiment, since the reading and writing of data are based on the mapping table entry in the dynamic random access memory, after the data writing, the mapping table entry in the dynamic random access memory will be changed, resulting in that the mapping table entry in the dynamic random access memory is different from the corresponding entry of the mapping table in the upper RAID volume, and the part of different data is defined as the dirty data in the embodiment.

[0069] In the embodiment, since the storage capacity of the dynamic random access memory is not completely persistent, but depends on the abnormal power-off persistent capability provided by the RAID card, only a small part of the mapping table data can be persistently saved, which is less than the space provided by the upper RAID volume for storing the mapping table. Therefore, it is necessary to regularly synchronize the mapping table entry in the dynamic random access memory to the mapping table in the upper RAID volume, and clean up the mapping table in the dynamic random access memory, so as to ensure that the latest mapping table entry can be persistently stored, guarantee the data security, and ensure the stable and effective operation of the Overlay volume, so as to ensure that the dirty data can be efficiently accessed in the dynamic random access memory, and improve the reading and writing efficiency.

[0070] In this embodiment, specifically, the preset dirty data size threshold can be determined according to a plurality of conditions, for example, it can be set according to the storage capacity of the dynamic random access memory, the update frequency of the upper RAID volume, the performance requirement of the Overlay volume, and other factors. This threshold aims to balance the requirement for persistence and read-write efficiency. When the dirty data in the dynamic random access memory reaches or exceeds this threshold, the dirty data is written back to the upper RAID volume, to ensure that the latest state of the mapping table can be persisted, thereby ensuring the stable operation of the foregoing steps 41 and 42, and improving the efficiency of file management based on the mapping table.

[0071] Optionally, in an embodiment, the preset dirty data size threshold is the smaller one of the storage size of the dynamic random access memory available for storing mapping table entries and the storage size that can be safely persisted when the dynamic random access memory is powered off.

[0072] In this optional embodiment, the storage size of the dynamic random access memory available for storing mapping table entries refers to the total storage capacity of the dynamic random access memory provided by the RAID card, which determines how many mapping table entries can be stored. The storage size that can be safely persisted when the dynamic random access memory is powered off refers to the minimum storage capacity that can ensure that data is not lost when an unexpected power failure or system failure occurs. For example, the storage size of the former is M1 bytes, and the storage size of the latter is M2 bytes, then the preset dirty data size threshold is M3 = Min(M1, M2), Min is the smaller one of the two values. When the dynamic random access memory capacity provided by the RAID card is large, but the power-off persistence capability of the RAID card or the system is limited, M1 can be greater than M2. When the power-off persistence capability of the RAID card is strong, but the random access memory capacity is limited, M2 can be greater than M1. By considering these two factors, the preset dirty data size threshold can ensure that the Overlay volume can maintain the integrity and latest state of the mapping table entries used in the read-write process in any case, while avoiding performance degradation caused by frequent write operations. In this way, the Overlay volume can optimize performance while ensuring data security, achieving efficient data management.

[0073] In this embodiment, when accessing a specific logical block address in the Overlay volume, the system first searches for the corresponding mapping table entry in the dynamic random access memory. If the mapping table entry of the target logical block address cannot be found in the dynamic random access memory, it usually means that the data has not been read into the dynamic random access memory or has been modified and written back to the upper RAID volume. In this case, and on the basis of determining that the data size of the dirty data in the dynamic random access memory reaches the preset dirty data size threshold, the corresponding mapping table entry needs to be read from the mapping table of the upper RAID volume to ensure that the logical block address can be correctly located. This process involves retrieving the corresponding mapping table entry from the mapping table in the upper RAID volume and loading it into the dynamic random access memory so that subsequent file read and write operations can be based on the latest mapping information. In this way, the Overlay volume can ensure that even in the face of frequent data access and modification, the accuracy and latest state of the mapping table entries available for use in the read and write process can be maintained, thereby ensuring the accuracy and reliability of file management and the efficiency of data access.

[0074] Optionally, referring to Figure 7 After determining that the data size of the dirty data in the dynamic random access memory reaches the preset dirty data size threshold and that the target logical block address is not queried from the mapping table entries of the dynamic random access memory, reading the corresponding mapping table entry from the upper RAID volume, and storing the corresponding mapping table entry into the dynamic random access memory, the method further includes:

[0075] Step 53: determining the least recently used degree of each mapping table entry in the dynamic random access memory based on the least recently used algorithm.

[0076] Step 54: storing the mapping table entries of the preset persistence size in the dynamic random access memory into the upper RAID volume in descending order of the least recently used degree, and the preset persistence size is equal to the size of the mapping table entry read from the upper RAID volume.

[0077] This embodiment provides a mapping table management strategy that optimizes the storage of mapping table entries in the dynamic random access memory through the least recently used algorithm (LRU). This strategy ensures that the least frequently used mapping table entries can be prioritized for elimination, thereby freeing up space for newly read mapping table entries. In this way, the Overlay volume can dynamically adjust the storage of the mapping table to adapt to different access patterns and data usage frequencies, thereby improving the overall efficiency of file management.

[0078] In this embodiment, the least recently used algorithm is a memory management algorithm that is based on the assumption that if a mapping table entry has not been accessed for a period of time, it is less likely to be accessed in the future. The least recently used degree calculated by this algorithm refers to the length of time that has passed since the mapping table entry was last accessed. In dynamic random access memory, according to the descending order of the least recently used degree, the mapping table entries that have not been accessed for the longest time will be placed at the front of the list. When it is necessary to remove the mapping table from the dynamic random access memory to free up space, the system will give priority to these least frequently used mapping table entries. In this way, the system can ensure that frequently accessed mapping table entries are retained in the fast access dynamic random access memory, while infrequently accessed mapping table entries are persisted to the upper RAID volume, thus ensuring access efficiency and accuracy, and optimizing memory usage.

[0079] In this embodiment, in the process of implementing the least recently used algorithm, the access of each mapping table entry is continuously tracked, and their least recently used degrees are updated in real time. When the mapping table entries in the dynamic random access memory reach the preset persistence size, the Overlay volume will write the least frequently used mapping table entries to the upper RAID volume according to the order of the least recently used degrees. This process is dynamic, meaning that as the file access pattern changes, the storage location of the mapping table entries will be adjusted accordingly to adapt to the current access requirements. In addition, by implementing the least recently used algorithm, the system can also reduce the performance loss caused by frequent reading of mapping table from the upper RAID volume. Since the least frequently used mapping table entries are preferentially eliminated, mapping table will only be read from the upper RAID volume when necessary, which can reduce the number of reads from the upper RAID volume, thereby improving the overall file management performance and ensuring that the Overlay volume can operate efficiently and stably even in the case of frequent mapping table entry updates.

[0080] In this embodiment, the mapping table is stored in a reserved space in the upper RAID volume, such as the end part of the upper RAID volume. Assuming that the range of the reserved empty logical block addresses is MAP TABLE L S to MAP TABLE LBA E, MAP TABLE LBA S is the starting logical block address stored in the mapping table, and MAP TABLE LBA E is the maximum logical block address of the reserved space of the mapping table. When it is necessary to read or persist the mapping table into the upper RAID volume, it is necessary to first determine the logical block address of the mapping table. Assuming that the block size of the logical block address in the current system, that is, the minimum granularity of reading and writing of a single logical block address, is B bytes, because 1 bit in the mapping table corresponds to one host data logical block address, the number of host data logical block addresses represented by one mapping table is D = B * 8, and assuming that the host data logical block address is D LBA, then the corresponding MAP TABLE LBA = D LBA / D + MAP TABLE LBA S. In practice, B bytes can be 4K bytes, and therefore the granularity of the mapping table for the purpose of persistence and reading from the upper RAID volume can be an integer multiple of 4K, and the specific value of the integer multiple is selected according to the specifications of the dynamic random access memory and other hardware limitations.

[0081] The embodiment realizes a mapping table management method through the above steps, which ensures the performance and stability of the Overlay volume when processing a large amount of data through efficient use of the dynamic random access memory and reasonable management of the upper RAID volume. Through the cache function of the dynamic random access memory, the Overlay volume can quickly respond to the read and write requests of the user, and at the same time, the mapping table is managed through the least recently used algorithm, which optimizes the use efficiency of the memory and reduces the frequent access to the upper RAID volume, thereby reducing the system delay and improving the overall performance. In addition, this method also considers the protection of the mapping table data in the case of power failure or system failure, and ensures that the mapping table data will not be lost even in unexpected situations, thereby ensuring the data security and stable operation of the Overlay volume.

[0082] Referring to Figure 8 , Figure 8 is a structural schematic diagram of an embodiment of the storage medium provided by the present application.

[0083] The storage medium 60 stores program data 61, and the program data 61, when executed by a processor, realizes the RAID card-based file management method as described in Figures 1 to 7 .

[0084] The program data 61 is stored in a storage medium 60, including a plurality of instructions for causing a network device (such as a router, a personal computer, a server, etc.) or a processor to execute all or part of the steps of the method of various embodiments of the present application.

[0085] Optionally, the storage medium 60 can be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a storage disk, or an optical disk, etc.

[0086] Referring to Figure 9 , Figure 9 is a structural schematic diagram of an embodiment of the computer device provided by the present application.

[0087] The computer device 70 includes a processor 72 and a memory 71 connected to each other, the memory 71 stores a computer program, and the processor 72 executes the computer program to implement the RAID card-based file management method as described in Figures 1 to 7 . The memory 71 can include the storage medium 60, or can be another separately developed memory.

[0088] Different from the prior art, the present application discloses a RAID card-based file management method, a storage medium, and a computer device. A lower RAID volume and an upper RAID volume of a newly created Overlay volume based on a RAID card are created; a basic common file is written into the lower RAID volume, and the data in the basic common file is associated with a corresponding logical block address; an initial mapping table is stored in the upper RAID volume, wherein the upper RAID volume is also used to store modified data of the basic common file, the mapping table is used to map the logical block addresses associated with the data in the Overlay volume and record the modification states of the logical block addresses. The logical block address level Overlay based on the RAID hardware is realized, so that the files stored in the Overlay volume can be managed at a high granularity, not only reducing unnecessary data replication and improving read-write efficiency, but also avoiding the space waste of the existing software-implemented file system, and because of the existence of the mapping table, the newly created Overlay volume can quickly locate and access data, improving the read-write performance of the Overlay volume and being beneficial to optimizing the overall performance of the host device. At the same time, the file management method does not need to depend on a specific operating system and kernel version, has good compatibility and flexibility, can adapt to file management requirements in different environments, and has wide applicability.

[0089] The various embodiments in the specification are described in progressive manner, and the same or similar parts among the various embodiments can be mutually referred to, and each embodiment focuses on the difference from other embodiments. In particular, for the storage medium embodiments and the computer device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0090] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

[0091] In several embodiments provided in the present application, it should be understood that the disclosed method, storage medium and computer device can be implemented in other ways. For example, the storage medium embodiments described above are merely illustrative, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0092] The units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment scheme.

[0093] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0094] The above is only an embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A file management method based on a RAID card, characterized in that, include: Create a new overlay volume based on the RAID card, including the lower-level RAID volume and the upper-level RAID volume; A basic common file is written into the lower-level RAID volume, and the data in the basic common file is associated with the corresponding logical block address; An initial mapping table is stored in the upper-layer RAID volume, wherein the upper-layer RAID volume is also used to store the modified data of the underlying public file, and the mapping table is used to map the logical block addresses associated with the data in the overlay volume and record the modification status of each logical block address; The mapping table entry corresponding to the logical block address is determined to be in the upper-layer RAID volume, and the mapping table entry is read from the mapping table of the upper-layer RAID volume into the dynamic random access memory; File reading and writing are performed based on the mapping table entries in the dynamic random access memory; If the mapping table entry in the dynamic random access memory is determined to be different from the corresponding mapping table entry in the upper-layer RAID volume, the mapping table entry in the dynamic random access memory is recorded as dirty data. If the size of the dirty data in the dynamic random access memory reaches a preset dirty data size threshold, and the target logical block address is not found in any of the mapping table entries in the dynamic random access memory, the corresponding mapping table entry is read from the upper-layer RAID volume, and the corresponding mapping table entry is stored in the dynamic random access memory.

2. The file management method based on a RAID card according to claim 1, characterized in that, The file management method also includes: In response to receiving a write command, the logical block address corresponding to the write command and the write data are confirmed; The write data is stored in the upper-layer RAID volume, and the modification status corresponding to the logical block address in the mapping table is updated.

3. The file management method based on a RAID card according to claim 1, characterized in that, The file management method also includes: In response to receiving a read command, confirm the logical block address corresponding to the read command; Based on the mapping table, determine the data to be read from the logical block address in the upper-layer RAID volume and / or the lower-layer RAID volume.

4. The file management method based on a RAID card according to claim 3, characterized in that, Based on the mapping table, determining the data to be read from the logical block address in the upper-layer RAID volume and / or the lower-layer RAID volume includes: Based on the mapping table, the modification state corresponding to the logical block address is determined as the initial state, and the data of the logical block address is read from the lower-level RAID volume into the temporary data storage area; Based on the mapping table, the modified state corresponding to the logical block address is determined to be a post-write state, and the data of the logical block address is read from the upper-layer RAID volume into the temporary data storage area.

5. The file management method based on a RAID card according to claim 1, characterized in that, After determining that the size of the dirty data in the dynamic random access memory (DRAM) reaches a preset dirty data size threshold, and no target logical block address is found in any of the mapping table entries of the DRAM, reading the corresponding mapping table condition from the upper-layer RAID volume, and storing the corresponding mapping table entry into the DRAM, the method further includes: Based on the Least Recently Used algorithm, the least recently used degree of each of the mapping table entries in the dynamic random access memory is determined; The mapping table entries in the dynamic random access memory are stored in the upper-layer RAID volume in descending order of the least recently used number of entries. The preset persistent size is equal to the size of the mapping table entries read from the upper-layer RAID volume.

6. The file management method based on a RAID card according to claim 1, characterized in that, The preset dirty data size threshold is the smaller of the storage size of the dynamic random access memory (DRAM) available to store the mapping table entries and the storage size of the DRAM that can be safely persisted when power is lost.

7. A storage medium storing program data thereon, characterized in that, When the program data is executed by the processor, it implements the steps of the file management method based on a RAID card as described in any one of claims 1 to 6.

8. A computer device, characterized in that, It includes an interconnected processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the steps of the file management method based on a RAID card as described in any one of claims 1 to 6.

Images