Windows file system real-time replication method, system and recovery method

Abstract

The present invention relates to a Windows file system real-time copy method, system and recovery method. The method comprises: a target directory acquisition step, a file copy and transmission step; a real-time monitoring step, a step of receiving and storing the copied file, a step of creating a file write record linked list, a step of capturing and sending I / O, and a step of receiving I / O and writing a log. The system comprises: a target path acquisition module, a real-time monitoring module, a first sending module, a first receiving module, a linked list node creation module, an I / O capture and storage module, an I / O data acquisition and stack space release module, an I / O data sending module, an I / O storage module and an I / O matching and log writing module. Based on the filter driver, the present invention combines a linked list and a plurality of structures to record and process various I / Os, and can perform real-time copying of designated directories and files under Windows.

Description

Technical Field

[0001] The invention belongs to the field of disaster recovery and backup, and relates to a Windows file system real-time replication method, system and recovery method. Background Art

[0002] Windows operating system is the general name for a series of operating systems developed and maintained by Microsoft. Windows operating system is widely used in the field of personal computers (PCs). It provides basic functions such as user interface, file management, multitasking, network connection, etc., and has become one of the preferred operating systems for many computer users.

[0003] Real-time replication generally refers to a data replication process in a computer system that can instantly copy data from one location to another. File system-based real-time replication focuses on data replication at the file level, and is usually used to ensure that data in the file system remains synchronized and updated in different locations or devices. Unlike general real-time replication based on the replication of the entire disk or partition, file system-based real-time replication focuses on a single file or folder. This approach allows finer control, such as synchronizing only specific directories or file types.

[0004] Currently, there is almost no technology for real-time replication of Windows file systems, and real-time replication has significant effects in the fields of data backup and recovery, disaster recovery, data sharing and synchronization.

[0005] Therefore, how to help users perform real-time copying of designated directories and files under Windows, and restore the historical status of these files, has become a technical problem that urgently needs to be solved. Summary of the invention

[0006] In order to solve the technical problems in the above background technology, the present invention provides a Windows file system real-time replication method, system and recovery method. The technical solution is as follows:

[0007] In a first aspect, a Windows file system real-time replication method is provided, which comprises a client program arranged at a production end user layer, a filter driver arranged at a production end kernel layer, and a server program arranged at a replication end user layer, the method comprising the steps of:

[0008] The client program selects a target directory and obtains the first copy file based on the target directory;

[0009] The client program monitors the target directory in real time;

[0010] The client program sends the first copy of the file to the server program;

[0011] The server program creates a replication root directory, then receives the first replication file, creates a replication subdirectory based on the received first replication file, and then puts the first replication file into the corresponding replication subdirectory;

[0012] The client program creates a new node of the file write record linked list. Each node of the file write record linked list includes: I / O file path, I / O write length, I / O write offset, write data and linked list pointer;

[0013] When the target directory sends an I / O request, the filter driver is started to capture the generated I / O and store the I / O data in the I / O data stack of the kernel layer of the production end;

[0014] The client program maps the I / O data in the I / O data stack to the production end user layer and writes the new node of the record linked list with a file for storage, and then releases the space of the I / O data stack;

[0015] The client program writes the I / O data of the file into the record linked list node and sends it to the server program;

[0016] The server program stores the received I / O data in a log write record structure, which consists of the I / O file name, timestamp, file size after I / O write, I / O write offset, I / O write size, write data, and index.

[0017] The server program finds the corresponding replication subdirectory according to the I / O file name in the log write record structure, and then stores the log write record structure containing the I / O data into the log file of the replication subdirectory.

[0018] In one embodiment, the client program monitors the target directory step in real time, including:

[0019] The client program creates a deletion file name array at the user level and records the file names of the files deleted in the target directory into the deletion file name array;

[0020] When a new file B is added to the target directory, the client program queries the file name of file B from the deleted file name array. If the deleted file name array does not contain the file name of file B, no additional processing is done and file B is treated as the first copy file; if the deleted file name array contains files A and B with the same name, the client program waits for the next instruction;

[0021] The client program starts to determine whether file B continues to use the real-time copy record of file A according to the instruction. If file B is restored by undoing the deletion operation of file A, the real-time copy record of file A will continue to be used; if file B is a newly added file with the same name as file A, the previous record of file A will not be used, and the client program will use file B as the first copy file.

[0022] In one of the embodiments, before the step of starting the filter driver to capture the generated I / O when the target directory sends an I / O request, and storing the I / O data in the I / O data stack of the kernel layer of the production end, the step further includes:

[0023] The client program creates a rename record structure at the user level. The rename record structure consists of an integer used to record the number of renamed files and two string arrays used to record the new and old file names respectively.

[0024] After the step of starting the filter driver to capture the generated I / O when the target directory sends an I / O request, and storing the I / O data in the I / O data stack of the kernel layer of the production end, the step further includes:

[0025] When the filter driver captures the rename I / O and stores it in the I / O data stack, the client program maps the new and old file names in the rename I / O from the I / O data stack to the user layer and stores them in the rename record structure;

[0026] The client program sends the rename record structure containing the new and old file names to the server program;

[0027] The server program receives and finds the corresponding copy subdirectory according to the old file name, and then changes the copy subdirectory name to the new file name.

[0028] It is important to understand that rename I / O is also part of modify I / O and will be captured by the driver normally, but for real-time replication on the replication side, different processing is required.

[0029] It is worth explaining that the reason for the special treatment of rename I / O is to match the files before and after the rename operation, so as to avoid the situation where the I / O generated by the new file name cannot match the copy subdirectory corresponding to the old file name on the copy side after the rename operation, resulting in the loss of the copy record before the file is renamed.

[0030] In one embodiment, the client program sends the I / O data of the file written into the record linked list node to the server program step, including:

[0031] The client program obtains I / O data from the node in the file write record linked list;

[0032] The client program constructs a transmission protocol structure according to the I / O data. The components of the transmission protocol structure are: I / O data packet type, I / O file root directory, I / O file subdirectory, I / O file size, I / O data length, and I / O data offset.

[0033] The client program transmits the transport protocol structure and I / O data to the server program in sequence;

[0034] The server program stores the received I / O data according to a log write record structure, where the log write record structure consists of an I / O file name, a timestamp, a file size after I / O write, an I / O write offset, an I / O write size, write data, and an index, including:

[0035] The server program receives the transport protocol structure and then receives the I / O data;

[0036] The server program performs integrity check on the I / O data according to the transmission protocol structure;

[0037] The server program stores the received I / O data according to the log write record structure. The log write record structure consists of the I / O file name, timestamp, file size after I / O write, I / O write offset, I / O write size, write data and index.

[0038] It should be understood that by constructing a transmission protocol based on I / O data and verifying the I / O data after transmission based on the transmission protocol, errors that occur during transmission can be discovered and reliability can be improved.

[0039] In one embodiment, after the server program finds the corresponding replication subdirectory according to the I / O file name in the log write record structure, and then stores the log write record structure containing the I / O data into the log file of the replication subdirectory, the server program further includes the following steps:

[0040] The server program calculates the survival time of each log file, and then compares the survival time and the recycling time, and deletes the logs whose survival time is greater than or equal to the recycling time, where the survival time value is the difference between the current system time and the last modification time of the file, and the recycling time value is a preset value.

[0041] In a second aspect, a Windows file system real-time replication system is provided, which has a client program arranged at a production end user layer, a filter driver arranged at a production end kernel layer, and a server program arranged at a replication end user layer, the system comprising:

[0042] The target path acquisition module is used to select the target directory and obtain the first copy file according to the target directory;

[0043] Real-time monitoring module, used to monitor the target directory in real time;

[0044] A first sending module, used for the client program to send the first copy file to the server program;

[0045] The first receiving module is used for the server program to create a replication root directory, then receive the first replication file, and create a replication subdirectory according to the received first replication file, and then put the first replication file into the corresponding replication subdirectory;

[0046] Link list node creation module, used by the client program to create a new node in the file write record linked list;

[0047] The I / O capture and storage module is used to start the filter driver to capture the generated I / O when the target directory sends an I / O request, and store the I / O data in the I / O data stack of the kernel layer of the production end;

[0048] The module for obtaining I / O data and releasing stack space is used by the client program to map the I / O data in the I / O data stack to the production end user layer and write the new node of the record linked list with a file for storage, and then release the space of the I / O data stack;

[0049] The I / O data sending module is used for the client program to send the I / O data written into the record linked list node to the server program;

[0050] The I / O storage module is used for the server program to store the received I / O data according to the log write record structure. The log write record structure consists of the I / O file name, timestamp, file size after I / O write, I / O write offset, I / O write size, write data and index;

[0051] The I / O matching and log writing module is used by the server program to find the corresponding replication subdirectory according to the I / O file name in the log writing record structure, and then store the log writing record structure containing the I / O data into the log file of the replication subdirectory.

[0052] In one embodiment, the real-time monitoring module includes:

[0053] The deletion file name array creation and recording unit is used for the client program to create a deletion file name array at the user layer and record the file names of the deleted files in the target directory into the deletion file name array;

[0054] The duplicate name judgment unit is used to query the file name of file B from the deleted file name array when a new file B is added to the target directory. If the deleted file name array does not contain the file name of file B, no additional processing is performed and file B is treated as the first copy file; if there are files A and B with the same name in the deleted file name array, wait for the next instruction;

[0055] The duplicate name processing unit is used for the client program to decide whether file B continues to use the real-time copy record of file A according to the instruction. If file B is restored from file A by undoing the deletion operation, the real-time copy record of file A will continue to be used; if file B is a newly added file with the same name as file A, the previous record of file A will not be used, and the client program will use file B as the first copy file.

[0056] In one embodiment, before the I / O capture and storage module, it also includes:

[0057] The renaming record structure creation module is used by the client program to create a renaming record structure at the user layer. The renaming record structure consists of an integer for recording the number of renamed files and two string arrays for recording the new and old file names respectively.

[0058] After the I / O capture and storage module, it also includes:

[0059] The rename I / O storage module is used for, when the filter driver captures the rename I / O and stores it in the I / O data stack, the client program maps the new and old file names in the rename I / O from the I / O data stack to the user layer and stores them in the rename record structure;

[0060] A rename record structure sending module is used for the client program to send the rename record structure containing the new and old file names to the server program;

[0061] The rename receiving and executing module is used for the server program to receive and find the corresponding copy subdirectory according to the old file name, and then modify the copy subdirectory name to the new file name.

[0062] In one embodiment, the I / O data sending module includes:

[0063] An I / O data acquisition unit is used for the client program to acquire I / O data from the node of the file write record linked list;

[0064] The protocol construction unit is used by the client program to construct a transmission protocol structure according to the I / O data. The components of the transmission protocol structure are: I / O data packet type, I / O file root directory, I / O file subdirectory, I / O file size, I / O data length, and I / O data offset;

[0065] The transmission unit is used for the client program to transmit the transmission protocol structure and I / O data to the server program in sequence;

[0066] The I / O storage module comprises:

[0067] The protocol and I / O data receiving unit is used for the server program to receive the transmission protocol structure and then receive the I / O data;

[0068] An I / O integrity check unit is used for the server program to perform integrity check on the I / O data according to the transmission protocol structure;

[0069] An I / O data storage unit is used for the server program to store the received I / O data according to a log write record structure, and the log write record structure consists of an I / O file name, a timestamp, a file size after I / O write, an I / O write offset, an I / O write size, write data, and an index;

[0070] In addition, after the I / O matching and log writing modules, it also includes:

[0071] The replication end space recovery module is used by the server program to calculate the survival time of each log file, and then compare the survival time and the recovery time, and delete the logs whose survival time is greater than or equal to the recovery time, where the survival time value is the difference between the current system time and the last modification time of the file, and the recovery time value is a preset value.

[0072] In a third aspect, a real-time recovery method for a Windows file system is provided, which is based on the above-mentioned real-time replication method for a Windows file system and further includes the steps of:

[0073] The server program selects the first copy file of the recovery target file, and then automatically matches and obtains the recovery log file corresponding to the recovery target file;

[0074] The server program parses the recovery log file to obtain a set of recovery time points;

[0075] The server program obtains the recovery time point T1, determines the I / O data at and before the recovery time point T1 according to the recovery log file, and then recovers the I / O data at and before the recovery time point T1 on the first copy file.

[0076] The beneficial effects of the present invention are:

[0077] 1. Based on the filter driver, the present invention combines multiple structures such as file record linked lists to capture and record I / O in the target directory, and performs corresponding processing on the copy end, thereby achieving a real-time copy effect of the target directory with high real-time copy efficiency;

[0078] 2. The present invention realizes real-time replication at the file system level. Unlike similar technologies that replicate volumes or partitions in real time, the present invention can replicate specific directories or files in real time, making it easier to precisely control the replication area while requiring lower network and storage costs.

[0079] 3. The present invention utilizes the kernel driver to capture I / O to directly obtain file changes without adding other overheads, which is more convenient and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0081] Figure 1 The figure is a flow chart of a Windows file system real-time replication method in Embodiment 1 of the present invention.

[0082] Figure 2 The figure is a logical structure diagram of the Windows file system real-time replication method in the first embodiment of the present invention.

[0083] Figure 3 The figure is a flow chart of a Windows file system real-time replication method in the second embodiment of the present invention.

[0084] Figure 4 The flowchart of the Windows file system real-time replication method in the third embodiment of the present invention.

[0085] Figure 5 This is a structural diagram of a Windows file system real-time replication system in Embodiment 4 of the present invention.

[0086] Figure 6 This is a structural diagram of the real-time monitoring module in the fourth embodiment of the present invention.

[0087] Figure 7 This is a structural diagram of the I / O data sending module in Embodiment 4 of the present invention.

[0088] Figure 8 This is a structural diagram of the I / O storage module in the fourth embodiment of the present invention.

[0089] Fig. 9The flowchart is a method for real-time recovery of a Windows file system in Embodiment 5 of the present invention.

[0090] In the accompanying drawings, the components represented by the reference numerals are listed as follows:

[0091] 4001, directory path acquisition module, 4002, real-time monitoring module, 4003, first sending module, 4004, first receiving module, 4005, linked list node creation module, 4006, rename record structure creation module, 4007, I / O capture and storage module, 4008, rename I / O storage module, 4009, rename record structure sending module, 4010, rename receiving and execution module, 4011, obtain I / O data and stack space release module, 4012, send I / O data module, 4013, I / O storage module, 4014, I / O matching and log writing module, 4015, copy end space recovery module, 40021, delete file name array creation and recording unit, 40022, duplicate name judgment unit, 40023, duplicate name processing unit, 40121, I / O data acquisition unit, 40122, construction protocol unit, 40123, transmission unit, 40131, protocol and I / O data receiving unit, 40132, I / O integrity check unit, 40133, I / O data storage unit. DETAILED DESCRIPTION

[0092] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0093] The method provided by the present invention has an application scope including but not limited to the following environments: the operating system is Windows 10, and the parsing process is written in C++ language.

[0094] Example 1

[0095] like Figure 1 As shown, a Windows file system real-time replication method is provided, which has a client program set at the user layer of the production end, a filter driver set at the kernel layer of the production end, and a server program set at the user layer of the replication end, and the method includes the steps of:

[0096] Step S001: The client program selects a target directory and obtains a first copy file according to the target directory.

[0097] It can be understood that the above-mentioned obtaining the first copy file according to the target directory specifically refers to: copying all files under the target directory to obtain the copy files of all files under the directory.

[0098] Step S002: the client program monitors the target directory in real time.

[0099] In addition, for ease of understanding, step S002 is described in detail below.

[0100] Optionally, the step S002 includes:

[0101] Step S0021, the client program creates a deletion file name array at the user layer, and records the file names of the deleted files in the target directory into the deletion file name array;

[0102] Step S0022, when a new file B is added to the target directory, the client program queries the file name of file B from the deleted file name array. If the deleted file name array does not contain the file name of file B, no additional processing is performed and file B is treated as the first copy file; if there are files A and B with the same name in the deleted file name array, wait for the next instruction;

[0103] Step S0023, the client program starts to determine whether file B continues to use the real-time copy record of file A according to the instruction. If file B is restored by undoing the deletion operation of file A, the real-time copy record of file A will continue to be used; if file B is a newly added file with the same name as file A, the previous record of file A will not be used, and the client program will use file B as the first copy file.

[0104] It is worth explaining that the judgment of whether the newly added file has the same name as the historical file is to ensure that the restored file after undoing the deletion can still retain the copy record before the deletion.

[0105] Step S003, the client program sends the first copy file to the server program.

[0106] It is understandable that the sending is performed using a network connection between the client program and the server program.

[0107] Step S004, the server program creates a replication root directory, then receives the first replication file, creates a replication subdirectory according to the received first replication file, and then puts the first replication file into the corresponding replication subdirectory.

[0108] Step S005, the client program creates a new node of the file write record linked list, each node of the file write record linked list includes: I / O file path, I / O write length, I / O write offset, write data and linked list pointer.

[0109] It is worth explaining that the I / O file path, I / O write length, I / O write offset, and write data are specifically:

[0110] I / O file path, that is, the location of the file to be written by the IO request packet in the operating system;

[0111] I / O write length, that is, the length of data to be written by the IO request packet;

[0112] I / O write offset, that is, the starting position where the IO request packet is to be written to the file;

[0113] Write data, that is, the data that the IO request packet specifically writes into the file.

[0114] Step S006, when the target directory sends an I / O request, the filter driver is started to capture the generated I / O, and the I / O data is stored in the I / O data stack of the kernel layer of the production end.

[0115] It is worth noting that the filter driver is a file filter driver based on the minifilter framework, which is used to monitor and process file system operations.

[0116] Step S007, the client program maps the I / O data in the I / O data stack to the production end user layer and writes the new node of the record linked list with a file for storage, and then releases the space of the I / O data stack.

[0117] It is worth noting that the I / O data stack belongs to the memory space and needs to be released in time after the data is retrieved, otherwise the Windows operating system will crash due to memory leaks.

[0118] Step S008, the client program writes the I / O data of the file into the record linked list node and sends it to the server program.

[0119] Optionally, in step S008, the file after sending the data is written into the record linked list node and released.

[0120] Releasing the file write record linked list nodes helps to utilize the space and prevent the file write record linked list space from expanding endlessly.

[0121] Step S009, the server program stores the received I / O data according to the log write record structure, which consists of the I / O file name, timestamp, file size after I / O write, I / O write offset, I / O write size, write data and index.

[0122] Step S010, the server program finds the corresponding replication subdirectory according to the I / O file name in the log write record structure, and then stores the log write record structure containing the I / O data into the log file of the replication subdirectory.

[0123] Step S011, the server program calculates the survival time of each log file, and then compares the survival time and the recycling time, and deletes the logs whose survival time is greater than or equal to the recycling time, where the survival time value is the difference between the current system time and the last modification time of the file, and the recycling time value is a preset value.

[0124] For ease of understanding, we provide a logical structure diagram of this technical solution, such as Figure 2 As shown, first, the user-layer client program on the production end transfers the first copy file to the server program, and the user-layer server program on the copy end creates a copy root directory and a corresponding copy subdirectory after receiving it; when the production APP modifies the file, the kernel-layer filter driver captures the I / O and stores it in the kernel-layer data stack, and the client program maps the data in the data stack to the user-layer file write record list and obtains data from it, and then transfers the I / O data to the server program, and the server program writes the I / O data as a log file to the corresponding copy subdirectory after receiving it.

[0125] Based on the filter driver, this embodiment combines multiple structures such as file record linked lists to capture and record I / O in the target directory, and performs corresponding processing on the copy end, thereby achieving a real-time copy effect of the target directory with high real-time copy efficiency; at the same time, it can copy the target directory in real time, and judge the situation of files with the same name as historical files in the target directory, so that files restored after undoing deletion can still retain the copy records before deletion, and at the same time, the space on the copy end is timely recovered, the overall situation is stable, and the real-time copy effect is good.

[0126] Example 2

[0127] like Figure 3 As shown, a Windows file system real-time replication method is provided, which has a client program set at the user layer of the production end, a filter driver set at the kernel layer of the production end, and a server program set at the user layer of the replication end, and the method includes the steps of:

[0128] Step S201: The client program selects a target directory and obtains a first-copy file according to the target directory.

[0129] Step S202: the client program monitors the target directory in real time.

[0130] Step S203: the client program sends the first copy file to the server program.

[0131] Step S204, the server program creates a replication root directory, then receives the first replication file, creates a replication subdirectory according to the received first replication file, and then puts the first replication file into the corresponding replication subdirectory.

[0132] Step S205, the client program creates a new node of the file write record linked list, each node of the file write record linked list includes: I / O file path, I / O write length, I / O write offset, write data and linked list pointer.

[0133] Step S206, when the target directory sends an I / O request, the filter driver is started to capture the generated I / O, and the I / O data is stored in the I / O data stack of the kernel layer of the production end.

[0134] Step S207, the client program maps the I / O data in the I / O data stack to the production end user layer and stores it in a new node of the record linked list using a file, and then releases the space of the I / O data stack.

[0135] Step S208, the client program obtains I / O data from the node of the file write record linked list.

[0136] Step S209, the client program constructs a transmission protocol structure according to the I / O data, and the constituent elements of the transmission protocol structure are: I / O data packet type, I / O file root directory, I / O file subdirectory, I / O file size, I / O data length, and I / O data offset.

[0137] It is worth explaining that the I / O data packet type, I / O file root directory, I / O file subdirectory, I / O file size, I / O data length, and I / O data offset are specifically:

[0138] I / O data packet type, that is, the operation type of the IO request packet, such as modification, rename, etc.;

[0139] I / O file root directory, that is, the first-level location of the file to be written by the IO request packet in the operating system;

[0140] I / O file subdirectory, that is, the non-first-level location of the file to be written by the IO request packet in the operating system;

[0141] I / O file size, that is, the size of the file after the IO request packet is written;

[0142] I / O data length, that is, the length of data to be written by the IO request packet;

[0143] I / O data offset, that is, the starting position where the IO request packet is to be written into the file.

[0144] Step S210, the client program transmits the transmission protocol structure and the I / O data to the server program in sequence, and writes the file after the data is retrieved into the space release of the record linked list node.

[0145] Step S211, the server program receives the transmission protocol structure and then receives the I / O data.

[0146] Step S212: the server program performs integrity check on the I / O data according to the transmission protocol structure.

[0147] Step S213, the server program stores the received I / O data according to the log write record structure, which consists of the I / O file name, timestamp, file size after I / O write, I / O write offset, I / O write size, write data and index.

[0148] Step S214, the server program finds the corresponding replication subdirectory according to the I / O file name in the log write record structure, and then stores the log write record structure containing the I / O data into the log file of the replication subdirectory.

[0149] Compared with Example 1, this embodiment performs transmission protocol construction and transmission before transmitting I / O data, which helps the server to verify the integrity of the received I / O data and improves the stability of data transmission.

[0150] Example 3

[0151] like Figure 4 As shown, a Windows file system real-time replication method is provided, which has a client program set at the user layer of the production end, a filter driver set at the kernel layer of the production end, and a server program set at the user layer of the replication end, and the method includes the steps of:

[0152] Step S301: The client program selects a target directory and obtains a first-copy file according to the target directory.

[0153] Step S302: the client program monitors the target directory in real time.

[0154] Step S303: the client program sends the first copy file to the server program.

[0155] Step S304, the server program creates a replication root directory, then receives the first replication file, creates a replication subdirectory according to the received first replication file, and then puts the first replication file into the corresponding replication subdirectory.

[0156] Step S305, the client program creates a new node of the file write record linked list, each node of the file write record linked list includes: I / O file path, I / O write length, I / O write offset, write data and linked list pointer.

[0157] Step S306, the client program creates a rename record structure at the user layer. The rename record structure consists of an integer for recording the number of renamed files and two string arrays for recording the new and old file names respectively.

[0158] Step S307, when the target directory sends an I / O request, the filter driver is started to capture the generated I / O, and the I / O data is stored in the I / O data stack of the kernel layer of the production end.

[0159] Step S308, the client program maps the I / O data in the I / O data stack to the production end user layer and writes the new node of the record linked list with a file for storage, and then releases the space of the I / O data stack.

[0160] Step S309, when the filter driver captures the rename I / O and stores it in the I / O data stack, the client program maps the new and old file names in the rename I / O from the I / O data stack to the user layer and stores them in the rename record structure.

[0161] Step S310, the client program sends the I / O data of writing the file into the record linked list node and the rename record structure containing the new and old file names to the server program.

[0162] Step S311, the server program receives data, and when a rename record structure is received, the corresponding copy subdirectory is found according to the old file name, and then the copy subdirectory name is modified to the new file name; when I / O data is received, the received I / O data is stored according to the log write record structure, and the log write record structure consists of the I / O file name, timestamp, file size after I / O write, I / O write offset, I / O write size, write data and index.

[0163] Step S312, the server program finds the corresponding replication subdirectory according to the I / O file name in the log write record structure, and then stores the log write record structure containing the I / O data into the log file of the replication subdirectory.

[0164] It is important to understand that although rename I / O is also part of modify I / O and will be captured by the driver normally, it needs to be handled differently for real-time replication on the replication end.

[0165] It is worth mentioning that the reason for the special treatment of rename I / O is to match the files before and after the rename operation, so as to avoid the situation where the I / O generated by the new file name cannot match the copy subdirectory corresponding to the old file name on the copy side after the rename operation, resulting in the loss of the copy record before the file is renamed.

[0166] Compared with Example 1, this embodiment performs special processing on the renaming I / O generated during the real-time copying of the target directory, so that the files and their I / O records after the renaming can match those before the renaming, maintaining the continuity of the real-time copying before and after the renaming operation, and the overall situation is stable and good.

[0167] Example 4

[0168] like Figure 5 As shown, in this embodiment, a Windows file system real-time replication system is provided, which has a client program set at the user layer of the production end, a filter driver set at the kernel layer of the production end, and a server program set at the user layer of the replication end, and the system includes:

[0169] The target path acquisition module 4001 is used to select a target directory and obtain the first copy file according to the target directory;

[0170] A real-time monitoring module 4002 is used to monitor the target directory in real time;

[0171] The first sending module 4003 is used for the client program to send the first copy file to the server program;

[0172] The first receiving module 4004 is used for the server program to create a replication root directory, then receive the first replication file, and create a replication subdirectory according to the received first replication file, and then put the first replication file into the corresponding replication subdirectory;

[0173] Link list node creation module 4005, used for the client program to create a new node in the file writing record linked list;

[0174] A rename record structure creation module 4006 is used for the client program to create a rename record structure at the user layer. The rename record structure consists of an integer for recording the number of renamed files and two string arrays for recording the new and old file names respectively.

[0175] The I / O capture and storage module 4007 is used to start the filter driver to capture the generated I / O when the target directory sends an I / O request, and store the I / O data in the I / O data stack of the kernel layer of the production end;

[0176] The rename I / O storage module 4008 is used for, when the filter driver captures the rename I / O and stores it in the I / O data stack, the client program maps the new and old file names in the rename I / O from the I / O data stack to the user layer and stores them by the rename record structure;

[0177] The rename record structure sending module 4009 is used for the client program to send the rename record structure storing the new and old file names to the server program;

[0178] The rename receiving and executing module 4010 is used for the server program to receive and find the corresponding copy subdirectory according to the old file name, and then modify the copy subdirectory name to the new file name;

[0179] The module 4011 for obtaining I / O data and releasing stack space is used for the client program to map the I / O data in the I / O data stack to the production end user layer and write the new node of the record linked list with a file for storage, and then release the space of the I / O data stack;

[0180] The I / O data sending module 4012 is used for the client program to send the I / O data written by the file in the record linked list node to the server program;

[0181] The I / O storage module 4013 is used for the server program to store the received I / O data according to the log write record structure, and the log write record structure consists of the I / O file name, timestamp, file size after I / O write, I / O write offset, I / O write size, write data and index;

[0182] I / O matching and log writing module 4014, used for the server program to find the corresponding replication subdirectory according to the I / O file name in the log writing record structure, and then store the log writing record structure containing the I / O data into the log file of the replication subdirectory;

[0183] The copy-end space recovery module 4015 is used by the server program to calculate the survival time of each log file, and then compare the survival time and the recovery time, and delete the logs whose survival time is greater than or equal to the recovery time, where the survival time value is the difference between the current system time and the last modification time of the file, and the recovery time value is a preset value.

[0184] Optional, such as Figure 6 As shown, the real-time monitoring module 4002 includes:

[0185] The deletion file name array creation and recording unit 40021 is used for the client program to create a deletion file name array at the user layer, and record the file names of the files deleted in the target directory into the deletion file name array;

[0186] The duplicate name judgment unit 40022 is used for when a new file B is added to the target directory, the client program searches the file name of file B from the deleted file name array. If the deleted file name array does not contain the file name of file B, no additional processing is performed and file B is treated as the first copy file; if there are files A and B with the same name in the deleted file name array, the next instruction is waited for;

[0187] The duplicate name processing unit 40023 is used for the client program to decide whether file B continues to use the real-time copy record of file A according to the instruction. If file B is restored from file A by undoing the deletion operation, the real-time copy record of file A continues to be used; if file B is a newly added file with the same name as file A, the previous record of file A is not used, and the client program uses file B as the first copy file.

[0188] Optional, such as Figure 7 As shown, the I / O data sending module 4012 includes:

[0189] The I / O data acquisition unit 40121 is used for the client program to acquire I / O data from the node of the file write record linked list;

[0190] The protocol construction unit 40122 is used for the client program to construct a transmission protocol structure according to the I / O data. The components of the transmission protocol structure are: I / O data packet type, I / O file root directory, I / O file subdirectory, I / O file size, I / O data length, and I / O data offset.

[0191] The transmission unit 40123 is used for the client program to transmit the transmission protocol structure and the I / O data to the server program in sequence;

[0192] Optional, such as Figure 8 As shown, the I / O storage module 4013,

[0193] The protocol and I / O data receiving unit 40131 is used for the server program to receive the transmission protocol structure and then receive the I / O data;

[0194] I / O integrity check unit 40132, used for the server program to perform integrity check on I / O data according to the transmission protocol structure;

[0195] The I / O data storage unit 40133 is used for the server program to store the received I / O data according to the log write record structure. The log write record structure consists of the I / O file name, timestamp, file size after I / O write, I / O write offset, I / O write size, write data and index.

[0196] This technical solution realizes real-time replication at the file system level. Unlike similar technologies that only replicate volumes or partitions in real time, this solution can replicate specific directories or files in real time, making it easier to precisely control the replication area while requiring lower network and storage costs.

[0197] Example 5

[0198] like Fig. 9As shown, in this embodiment, a real-time recovery method for a Windows file system is provided, which is based on the real-time replication method for a Windows file system described in any one of Embodiments 1 to 3 above, and further includes the steps of:

[0199] Step S501, the server program selects the first copy file of the recovery target file, and then automatically matches and obtains the recovery log file corresponding to the recovery target file;

[0200] Step S502, the server program parses the recovery log file to obtain a recovery time point set;

[0201] Step S503, the server program obtains the recovery time point T1, and determines the I / O data at and before the recovery time point T1 according to the recovery log file, and then recovers the I / O data at and before the recovery time point T1 on the first copy file.

[0202] It is worth noting that the automatic matching and obtaining of the recovery log file corresponding to the recovery target file in step S501 specifically refers to: matching the log file having the same path and the same conventional file name as the first-copy file.

[0203] It needs to be explained that the file name includes the regular name and the extension name. For example, A.txt, A is the regular name of the file, which is usually assigned by humans, and txt is the file extension name, which determines the file type and is generally determined by the software.

[0204] It is also worth mentioning that in step S502, the server program parses the recovery log file to obtain a set of recovery time points, which specifically means that the log file is generated by the above-mentioned Windows file system real-time replication method, and each time it is modified in the format of a log write record structure. When parsing, it is divided according to the index in the log write structure to obtain a set of recovery points.

[0205] The embodiment of the present invention restores the copied file to various historical states, and the restoration result is consistent with the historical file state, with a stable effect.

[0206] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A Windows file system real-time replication method, characterized in that: A method having a client program arranged at a production end user layer, a filter driver at a production end kernel layer, and a server program arranged at a replication end user layer comprises the steps of: The client program selects a target directory and obtains the first copy file based on the target directory; The client program monitors the target directory in real time; The client program sends the first copy of the file to the server program; The server program creates a replication root directory, then receives the first replication file, creates a replication subdirectory based on the received first replication file, and then puts the first replication file into the corresponding replication subdirectory; The client program creates a new node for file writing. Each node in the file writing record list includes: I / O file path, I / O writing length, I / O writing offset, writing data and linked list pointer; When the target directory sends an I / O request, the filter driver is started to capture the generated I / O and store the I / O data in the I / O data stack of the kernel layer of the production end; The client program maps the I / O data in the I / O data stack to the production end user layer and writes the new node of the record linked list with a file for storage, and then releases the space of the I / O data stack; The client program sends the I / O data written into the record linked list node of the file to the server program, and releases the file written into the record linked list node after sending the data; The server program stores the received I / O data in a log write record structure, which consists of the I / O file name, timestamp, file size after I / O write, I / O write offset, I / O write size, write data, and index. The server program finds the corresponding replication subdirectory according to the I / O file name in the log write record structure, and then stores the log write record structure containing the I / O data into the log file of the replication subdirectory.

2. The Windows file system real-time replication method according to claim 1, characterized in that: The step of the client program monitoring the target directory in real time also includes: The client program creates a deletion file name array at the user level and records the file names of the files deleted in the target directory into the deletion file name array; When a new file B is added to the target directory, the client program queries the file name of file B from the deleted file name array. If the deleted file name array does not contain the file name of file B, no additional processing is done and file B is treated as the first copy file; if the deleted file name array contains files A and B with the same name, the client program waits for the next instruction; The client program starts to determine whether file B continues to use the real-time copy record of file A according to the instruction. If file B is restored by undoing the deletion operation of file A, the real-time copy record of file A will continue to be used; if file B is a newly added file with the same name as file A, the previous record of file A will not be used, and the client program will use file B as the first copy file.

3. The Windows file system real-time replication method according to claim 1, characterized in that: The step of sending the I / O data written by the client program to the server program includes: The client program obtains I / O data from the node in the file write record linked list; The client program constructs a transmission protocol structure according to the I / O data. The components of the transmission protocol structure are: I / O data packet type, I / O file root directory, I / O file subdirectory, I / O file size, I / O data length, and I / O data offset. The client program transmits the transport protocol structure and I / O data to the server program in sequence; The server program stores the received I / O data according to a log write record structure, where the log write record structure consists of an I / O file name, a timestamp, a file size after I / O write, an I / O write offset, an I / O write size, write data, and an index, including: The server program receives the transport protocol structure and then receives the I / O data; The server program performs integrity check on the I / O data according to the transmission protocol structure; The server program stores the received I / O data according to the log write record structure. The log write record structure consists of the I / O file name, timestamp, file size after I / O write, I / O write offset, I / O write size, write data and index.

4. The Windows file system real-time replication method according to claim 1, characterized in that: Before the step of starting the filter driver to capture the generated I / O when the target directory sends an I / O request and storing the I / O data in the I / O data stack of the kernel layer of the production end, the step further includes: The client program creates a rename record structure at the user level. The rename record structure consists of an integer used to record the number of renamed files and two string arrays used to record the new and old file names respectively. After the step of starting the filter driver to capture the generated I / O when the target directory sends an I / O request, and storing the I / O data in the I / O data stack of the kernel layer of the production end, the step further includes: When the filter driver captures the rename I / O and stores it in the I / O data stack, the client program maps the new and old file names in the rename I / O from the I / O data stack to the user layer and stores them in the rename record structure; The client program sends the rename record structure containing the new and old file names to the server program; The server program receives and finds the corresponding copy subdirectory according to the old file name, and then changes the copy subdirectory name to the new file name.

5. The Windows file system real-time replication method according to claim 1, characterized in that: After the server program finds the corresponding replication subdirectory according to the I / O file name in the log write record structure, and then stores the log write record structure containing the I / O data into the log file of the replication subdirectory, the server program also includes the following steps: The server program calculates the survival time of each log file, and then compares the survival time and the recycling time, and deletes the logs whose survival time is greater than or equal to the recycling time, where the survival time value is the difference between the current system time and the last modification time of the file, and the recycling time value is a preset value.

6. A Windows file system real-time replication system, characterized in that: The system comprises a client program arranged at a production end user layer, a filter driver arranged at a production end kernel layer, and a server program arranged at a replication end user layer, and includes: The target path acquisition module is used for the client program to select the target directory and obtain the first copy file according to the target directory; Real-time monitoring module, used by client programs to monitor the target directory in real time; A first sending module, used for the client program to send the first copy file to the server program; The first receiving module is used for the server program to create a replication root directory, then receive the first replication file, and create a replication subdirectory according to the received first replication file, and then put the first replication file into the corresponding replication subdirectory; Link list node creation module, used by the client program to create a new node in the file write record linked list; The I / O capture and storage module is used to start the filter driver to capture the generated I / O when the target directory sends an I / O request, and store the I / O data in the I / O data stack of the kernel layer of the production end; The module for obtaining I / O data and releasing stack space is used by the client program to map the I / O data in the I / O data stack to the production end user layer and write the new node of the record linked list with a file for storage, and then release the space of the I / O data stack; The I / O data sending module is used for the client program to send the I / O data written into the record linked list node to the server program, and release the file written into the record linked list node after the data is sent; The I / O storage module is used for the server program to store the received I / O data according to the log write record structure. The log write record structure consists of the I / O file name, timestamp, file size after I / O write, I / O write offset, I / O write size, write data and index; The I / O matching and log writing module is used by the server program to find the corresponding replication subdirectory according to the I / O file name in the log writing record structure, and then store the log writing record structure containing the I / O data into the log file of the replication subdirectory.

7. The Windows file system real-time replication system according to claim 6, characterized in that: The real-time monitoring module further includes: The deletion file name array creation and recording unit is used for the client program to create a deletion file name array at the user layer and record the file names of the deleted files in the target directory into the deletion file name array; The duplicate name judgment unit is used to query the file name of file B from the deleted file name array when a new file B is added to the target directory. If the deleted file name array does not contain the file name of file B, no additional processing is performed and file B is treated as the first copy file; if there are files A and B with the same name in the deleted file name array, wait for the next instruction; The duplicate name processing unit is used for the client program to decide whether file B continues to use the real-time copy record of file A according to the instruction. If file B is restored from file A by undoing the deletion operation, the real-time copy record of file A will continue to be used; if file B is a newly added file with the same name as file A, the previous record of file A will not be used, and the client program will use file B as the first copy file.

8. The Windows file system real-time replication system according to claim 6, characterized in that: The I / O data sending module comprises: An I / O data acquisition unit is used for the client program to acquire I / O data from the node of the file write record linked list; The protocol construction unit is used by the client program to construct a transmission protocol structure according to the I / O data. The components of the transmission protocol structure are: I / O data packet type, I / O file root directory, I / O file subdirectory, I / O file size, I / O data length, and I / O data offset; The transmission unit is used for the client program to transmit the transmission protocol structure and I / O data to the server program in sequence; The I / O storage module comprises: The protocol and I / O data receiving unit is used for the server program to receive the transmission protocol structure and then receive the I / O data; An I / O integrity check unit is used for the server program to perform integrity check on the I / O data according to the transmission protocol structure; The I / O data storage unit is used for the server program to store the received I / O data according to the log write record structure. The log write record structure consists of the I / O file name, timestamp, file size after I / O write, I / O write offset, I / O write size, write data and index.

9. The Windows file system real-time replication system according to claim 6, characterized in that: Before the I / O capture and storage module, it also includes: The renaming record structure creation module is used by the client program to create a renaming record structure at the user layer. The renaming record structure consists of an integer for recording the number of renamed files and two string arrays for recording the new and old file names respectively. After the I / O capture and storage module, it also includes: The rename I / O storage module is used for, when the filter driver captures the rename I / O and stores it in the I / O data stack, the client program maps the new and old file names in the rename I / O from the I / O data stack to the user layer and stores them in the rename record structure; A rename record structure sending module is used for the client program to send the rename record structure containing the new and old file names to the server program; The rename receiving and executing module is used for the server program to receive and find the corresponding copy subdirectory according to the old file name, and then change the copy subdirectory name to the new file name; In addition, after the I / O matching and log writing modules, it also includes: The replication end space recovery module is used by the server program to calculate the survival time of each log file, and then compare the survival time and the recovery time, and delete the logs whose survival time is greater than or equal to the recovery time, where the survival time value is the difference between the current system time and the last modification time of the file, and the recovery time value is a preset value.

10. A real-time recovery method for Windows file system, characterized in that: The Windows file system real-time replication method according to any one of claims 1 to 5 further comprises the steps of: The server program selects the first copy file of the recovery target file, and then automatically matches and obtains the recovery log file corresponding to the recovery target file; The server program parses the recovery log file to obtain a set of recovery time points; The server program obtains the recovery time point T1, determines the I / O data at and before the recovery time point T1 according to the recovery log file, and then recovers the I / O data at and before the recovery time point T1 on the first copy file.

Images