A fast and accurate restoration method for dynamic files based on NFS

Abstract

The present invention belongs to the field of data processing technology, and discloses a fast and accurate restoration processing method for dynamic files based on NFS. The specific operation steps are as follows: the first step is to input the data to be processed into a database to be processed; the second step is to then distribute the data system in the database to be processed to various hierarchical processing programs. The present invention pre-blocks the file data, then sets the corresponding limit value, and adopts a staged file processing method to effectively avoid the traditional centralized processing situation. This patent implements the NFS protocol based on the flow of file restoration process, does not need to cache a large number of files, and is pre-set with multiple docking devices to divide the huge data files into multiple files, and simultaneously speed up its work efficiency. Finally, when the corresponding summary program is set, the pre-processed partial files will be spliced ​​together, and the final output is a complete and effective restoration file.

Description

Technical Field

[0001] The present invention belongs to the technical field of data restoration, and in particular is a fast and accurate restoration processing method for dynamic files based on NFS. Background Art

[0002] NFS is the abbreviation of Network File System. Its biggest function is to allow different machines and different operating systems to share files (sharefiles) with each other through the network. The directory of the remote host can be mounted through NFS, and accessing the directory is just like accessing a local directory, so it can also be simply regarded as a file server (FileServer). During the process of file operations, users often lose file data for some reasons. In this era of network data, the loss of some important data is undoubtedly a fatal mistake.

[0003] In response to the above situation, people have also developed corresponding file restoration methods. The existing NFS file restoration technology mainly uses a large cache for file restoration, applies for a sufficiently large cache, and sets a limit. Files exceeding the set limit are discarded, and files smaller than the limit are cached. The data blocks of the files are sorted in the cache, and the md5 is calculated to restore the files. This processing method is too simple and imperfect, and the processing is often not accurate and thorough, and file residues often appear.

[0004] In order to solve this problem, this patent proposes a fast and accurate restoration method for dynamic files based on NFS. Summary of the invention

[0005] The purpose of the present invention is to solve the above problems. The present invention provides a fast and accurate restoration processing method for dynamic files based on NFS, which has the advantages of realizing data block cache sorting similar to sliding windows for data blocks, and can restore NFS-transmitted files of any size using only a small amount of cache, thereby reducing memory occupancy and improving file restoration efficiency.

[0006] To achieve the above purpose, the present invention provides the following technical solution: a fast and accurate restoration method for dynamic files based on NFS, and the specific operation steps are as follows:

[0007] The first step is to input the data to be processed into the database to be processed;

[0008] In the second step, the data in the database to be processed are then systematically allocated to various hierarchical processing procedures;

[0009] 1) Set the cache size;

[0010] 2) Mark the data block number to be received;

[0011] 3) Officially accept the file data block and determine whether the data block number is sorted:

[0012] If no, return to step 2) and repeat the operation; if yes, proceed to the next stage of operation;

[0013] 4) Sorting files for sending and processing;

[0014] 5) Determine whether the data block is the last block of data:

[0015] If no, return to step 2) and repeat the operation; if yes, proceed to the next stage of operation;

[0016] In the third step, the operation is completed and the processed data is collected and processed;

[0017] The fourth step is to collect the processing results of the third step together for comparison and analysis to obtain the final result;

[0018] Step 5: The final restored data is output and processed;

[0019] Although NFS data blocks are not transmitted in sequence, there is a certain range of disorder, which will not exceed 2M. A 2M cache is set to cache and sort the received data blocks. When the cached data blocks are in the correct order, they can be sent to the restore program and md5 (update md5) is calculated. Then the cache contents are cleared to receive new data blocks, and caching and sorting are continued until the file is received.

[0020] As a preferred technical solution of the present invention, the data block number is obtained by numbering the data blocks, and the data blocks are obtained by decomposing the files.

[0021] As a preferred technical solution of the present invention, the memory value of the data block is less than the set cache value, and can enter the hierarchical processing program for operation and processing. A large file is divided into several small parts, and a method similar to stage-by-stage processing is adopted to reduce the workload of the equipment. For example, the capacity of a landfill is fixed. If the garbage is processed directly at once, a considerable part of the materials will often be in the waiting stage. At the same time, due to the large processing volume, it is easy to have unclean processing. This patent sets the landfill into multiple blocks and operates synchronously. The waiting time of the materials does not change much, but the efficiency and effect of garbage processing are greatly improved. Similarly, the data processing situation of this patent is similar. In order to solve the data waiting situation, we have added many hierarchical processing programs to operate synchronously, which minimizes the waiting time and optimizes the working environment.

[0022] As a preferred technical solution of the present invention, NFS data block transmission adopts out-of-order transmission, and the limited file memory value of the out-of-order transmission is smaller than the set cache value.

[0023] As a preferred technical solution of the present invention, the main operation blocks of NFS file restoration are divided into a client processing end, a network information end and a system service end. The client processing end is divided into an enterprise port and a personal port. The personal port and the enterprise port often process different amounts of data, so the number of hierarchical processing programs required is different. The number of hierarchical processing programs is floating and can be adjusted according to actual conditions. The basic data retrieval reference Figure 3 That's it.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. The present invention pre-blocks the file data, then sets the corresponding limit value, and adopts a staged file processing method to effectively avoid the traditional centralized processing situation. This patent implements the NFS protocol based on the flow of file restoration process, does not need to cache a large number of files, only uses a 2M cache, and completes the accurate restoration of files of any size transmitted by the NFS protocol. In addition, in order to ensure the rapid processing of data, multiple docking devices are pre-set to divide the huge data files into multiple ones, and simultaneously speed up their work efficiency. Finally, when the corresponding aggregation program is set, the pre-processed partial files will be spliced ​​together, and the final output is a complete and effective restored file. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the overall workflow of the present invention;

[0027] Figure 2 It is a schematic diagram of the hierarchical processing procedure of the present invention;

[0028] Figure 3 This is a schematic diagram of the file restoration path of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] like Figures 1 to 3 As shown, the present invention provides a fast and accurate restoration processing method for dynamic files based on NFS, and the specific operation steps are as follows:

[0031] The first step is to input the data to be processed into the database to be processed;

[0032] In the second step, the data in the database to be processed are then systematically allocated to various hierarchical processing procedures;

[0033] 1) Set the cache size;

[0034] 2) Mark the data block number to be received;

[0035] 3) Officially accept the file data block and determine whether the data block number is sorted:

[0036] If no, return to step 2) and repeat the operation; if yes, proceed to the next stage of operation;

[0037] 4) Sorting files for sending and processing;

[0038] 5) Determine whether the data block is the last block of data:

[0039] If no, return to step 2) and repeat the operation; if yes, proceed to the next stage of operation;

[0040] In the third step, the operation is completed and the processed data is collected and processed;

[0041] The fourth step is to collect the processing results of the third step together for comparison and analysis to obtain the final result;

[0042] Step 5: The final restored data is output and processed;

[0043] Although NFS data blocks are not transmitted in sequence, there is a certain range of disorder, which will not exceed 2M. A 2M cache is set to cache and sort the received data blocks. When the cached data blocks are in the correct order, they can be sent to the restore program and md5 (update md5) is calculated. Then the cache contents are cleared to receive new data blocks, and caching and sorting are continued until the file is received.

[0044] The data block number is a numbering process for the data block, and the data block is obtained by decomposing the file.

[0045] Among them, the memory value of the data block is less than the set cache value, and it can enter the hierarchical processing program for operation and processing. Dividing a large file into several small parts, using a method similar to stage-by-stage processing, can reduce the workload of the equipment. For example, the capacity of a landfill is fixed. If the garbage is processed directly at once, a considerable part of the material will often be in the waiting stage. At the same time, due to the large processing volume, it is easy to cause unclean processing. This patent sets the landfill into multiple blocks and operates synchronously. The waiting time of the materials does not change much, but the efficiency and effect of garbage processing are greatly improved. Similarly, the data processing situation of this patent is similar. In order to solve the data waiting situation, we have added many hierarchical processing programs to operate synchronously, which minimizes the waiting time and optimizes the working environment.

[0046] Among them, NFS data block transmission adopts out-of-order transmission, and the limited file memory value of the out-of-order transmission is smaller than the set cache value.

[0047] Among them, the main operation blocks of NFS file restoration are divided into client processing end, network information end and system service end. The client processing end is divided into enterprise and personal ports. The personal port and the enterprise port often process different amounts of data, so the number of hierarchical processing programs required is different. The number of hierarchical processing programs is floating and can be adjusted according to actual conditions. The basic data retrieval reference Figure 3 That's it.

[0048] Embodiment 1

[0049] If the file test.txt is 5M in size, it is divided into 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 during transmission, each block is 0.5M, and the transmission order is 2, 3, 1, 5, 4, 6, 8, 7, 10, 9;

[0050] A. Apply for setting 2M memory;

[0051] B. The data block to be received at the beginning of the mark is data block 1. Data block 2 is received first, not the first block of data, and is cached;

[0052] Data block 3 is received, which is not the first data buffer;

[0053] Receive data block 1, which is the first block of data. Sort the cached data blocks 1, 2, and 3 in order, update the md5 value, send it, and update the next data block to be received to 4;

[0054] Clear the cache, continue receiving data, and sort the cache;

[0055] C. Data reception is completed and md5 value calculation is completed;

[0056] After the data is produced, it is compared and summarized, and finally output in a restored state.

[0057] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0058] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A fast and accurate restoration method for dynamic files based on NFS, characterized in that: The specific steps are as follows: The first step is to input the data to be processed into the database to be processed; In the second step, the data in the database to be processed are then systematically allocated to various hierarchical processing procedures; 1) Set the cache size; 2) Mark the data block number to be received; 3) Officially accept the file data block and determine whether the data block number is sorted: If no, return to step 2) and repeat the operation; if yes, proceed to the next stage of operation; 4) Sorting files for sending and processing; 5) Determine whether the data block is the last block of data: If no, return to step 2) and repeat the operation; if yes, proceed to the next stage of operation; In the third step, the operation is completed and the processed data is collected and processed; The fourth step is to collect the processing results of the third step together for comparison and analysis to obtain the final result; In the fifth step, the finally restored data is output and processed.

2. The NFS-based dynamic file fast and accurate restoration processing method according to claim 1 is characterized in that: The data block number is a numbering process for the data block, and the data block is obtained by decomposing the file.

3. The method for fast and accurate restoration of dynamic files based on NFS according to claim 1, characterized in that: The memory value of the data block is less than the set cache value and can enter the hierarchical processing program for operation and processing.

4. The method for fast and accurate restoration of dynamic files based on NFS according to claim 1, characterized in that: NFS data block transmission uses out-of-order transmission, and the limited file memory value of the out-of-order transmission is smaller than the set cache value.

5. The method for fast and accurate restoration of dynamic files based on NFS according to claim 1, characterized in that: The main operation blocks of NFS file restoration are divided into client processing end, network information end and system service end. The client processing end is divided into enterprise and personal ports.

Images