Storage control device, storage control method, and storage control program

Abstract

To reduce computational loads of a file management system.SOLUTION: A storage control device is configured to: compare a secondary hash value calculated from a data file written to a storage device with a secondary hash value included in file information stored in a directory; if the secondary hash value of the data file written to the storage device does not match the secondary hash value included in the file information stored in the directory, read a secondary hash entry corresponding to the secondary hash value of the data file written to the storage device from an update-only hash table; and execute processing of updating the file information using a primary hash value and hash entry address of the most recent data file included in the secondary hash entry, and a physical address of the most recent data file.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a storage control device, a storage control method, and a storage control program, and, for example, to a storage control device, a storage control method, and a storage control program that implement a deduplication function. [Background technology]

[0002] In a VDI (Virtual Desktop Infrastructure) environment, a large number of users use a common OS (Operating System) and common applications on virtual PCs (Personal Computers).

[0003] In an OS update in a VDI environment, the update operation is performed for each user on a common OS image. If the OS image before the update in each user's VDI environment is common, the OS image after the update will also be common among users.

[0004] If the storage device that stores the VDI environment has a deduplication function, related technologies process each user's operating system and applications as duplicate data, thereby improving storage capacity efficiency and speeding up read processing. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2010-100733 Summary of the Invention [Problem to be solved by the invention]

[0006] In related technology, each time a user using a VDI environment performs an update operation, a hash value of the updated data is calculated, and the hash values ​​are compared, duplicates are eliminated, and the number of duplicates of old OS images is subtracted.

[0007] The hash value of data requires a certain bit length, but as the bit length of the hash value increases, the computational load of the hash value increases exponentially, resulting in a long time required for deduplication update processing.

[0008] The present disclosure has been made in view of the above-mentioned problems, and its purpose is to reduce the computational load of a file management system. [Means for solving the problem]

[0009] A storage control device according to one aspect of the present disclosure is a storage control device that controls a storage device, the storage device being connected to a directory, a hash table, an update-only hash table, and a disk, actual data files are saved on the disk, the directory stores file information including the physical address of the actual data file, a main hash value used in deduplication processing of the actual data file, and a secondary hash value having a bit length shorter than the main hash value, the hash table stores hash entries and a hash entry search database for searching the hash entries from the main hash values, the update-only hash table stores secondary hash entries corresponding to the secondary hash values, the secondary hash entries include the main hash value (new) and hash entry address of the latest data file, the main hash value (old) of the data file before update, and the physical address of the latest data file, and the secondary hash value calculated from the data file written to the storage device and the the secondary hash value included in the file information stored in the directory, and if the secondary hash value of the data file written to the storage device does not match the secondary hash value included in the file information stored in the directory, the secondary hash entry corresponding to the secondary hash value of the data file written to the storage device is read from the update-only hash table, and the main hash value (old) of the data file before the update is compared with the main hash value before the update recorded in the directory using a sub-entry included in the secondary hash entry, and if a match is detected between the main hash value (old) of the data file before the update and the main hash value recorded in the directory, the data file written to the storage device is compared with the latest data file, and if a match is confirmed between the data file written to the storage device and the latest data file, it is determined to be duplicate data, and the main hash value and hash entry address of the latest data file, andA process of updating the file information is performed using the physical address of the latest data file.

[0010] A storage control method according to one aspect of the present disclosure is a storage control method for controlling a storage device, the storage device being connected to a directory, a hash table, an update-only hash table, and a disk, actual data files are saved on the disk, the directory stores file information including a physical address of the actual data file, a main hash value used in deduplication processing of the actual data file, and a secondary hash value having a bit length shorter than the main hash value, the hash table stores hash entries and a hash entry search database for searching the hash entries from the main hash values, the update-only hash table stores secondary hash entries corresponding to the secondary hash values, the secondary hash entries include the main hash value (new) and hash entry address of a latest data file, the main hash value (old) of a data file before update, and the physical address of the latest data file, and the storage control method a process of comparing a secondary hash value calculated from the data file written to the storage device with the secondary hash value included in the file information stored in the directory; if the secondary hash value of the data file written to the storage device does not match the secondary hash value included in the file information stored in the directory, reading from the update-only hash table the secondary hash entry corresponding to the secondary hash value of the data file written to the storage device, and comparing the main hash value (old) of the data file before the update with the main hash value before the update recorded in the directory based on a sub-entry included in the secondary hash entry; if a match is detected between the main hash value (old) of the data file before the update and the main hash value recorded in the directory, comparing the data file written to the storage device with the latest data file; and if a match is confirmed between the data file written to the storage device and the latest data file, determining that the data is duplicate data;and updating the file information using the main hash value and the hash entry address of the latest data file, and the physical address of the latest data file.

[0011] A storage control program according to one aspect of the present disclosure is a storage control program for controlling a storage device, the storage device being connected to a directory, a hash table, an update-only hash table, and a disk, actual data files being saved on the disk, the directory storing file information including the physical address of the actual data file, a main hash value used in deduplication processing of the actual data file, and a secondary hash value having a bit length shorter than the main hash value, the hash table storing hash entries and a hash entry search database for searching the hash entries from the main hash values, the update-only hash table storing secondary hash entries corresponding to the secondary hash values, the secondary hash entries including the main hash value (new) and hash entry address of the latest data file, the main hash value (old) of the data file before update, and the physical address of the latest data file, a process of comparing a secondary hash value calculated from a data file written to the storage device with the secondary hash value included in the file information stored in the directory; if the secondary hash value of the data file written to the storage device does not match the secondary hash value included in the file information stored in the directory, reading from the update-only hash table the secondary hash entry corresponding to the secondary hash value of the data file written to the storage device, and comparing the main hash value (old) of the data file before the update with the main hash value before the update stored in the directory based on a sub-entry included in the secondary hash entry; if a match is detected between the main hash value (old) of the data file before the update and the main hash value stored in the directory, comparing the data file written to the storage device with the latest data file; and if a match is confirmed between the data file written to the storage device and the latest data file, determining that the data is duplicate data;and updating the file information using the main hash value and the hash entry address of the latest data file, and the physical address of the latest data file. [Effects of the Invention]

[0012] According to one aspect of the present disclosure, it is possible to perform deduplication update processing at high speed. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating a configuration of a file system according to an embodiment. [Figure 2] 1 is a block diagram illustrating a configuration of a storage control device according to an embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of a directory structure according to an embodiment. [Figure 4] FIG. 1 illustrates an example of a hash table structure according to one embodiment. [Figure 5] FIG. 10 illustrates an example of a hash entry search database structure according to one embodiment. [Figure 6] FIG. 10 illustrates an example of an update-only hash table structure according to one embodiment. [Figure 7] 10A and 10B are diagrams illustrating an example of the structure of a secondary hash entry and an additional secondary hash entry according to an embodiment. [Figure 8] FIG. 10 shows the directory structure and secondary hash entries before all users have updated their data files. [Figure 9] FIG. 10 shows the directory structure and secondary hash entries after only one user has updated the data file. [Figure 10] FIG. 10 shows the directory structure and secondary hash entries after a second user updates the data file. [Figure 11] FIG. 10 shows the directory structure and secondary hash entries after all users have updated their data files. [Figure 12] 10 is a flowchart illustrating a part of the flow of processing executed by a storage control device of a file system according to an embodiment. [Figure 13] 10 is a flowchart illustrating a part of the flow of processing executed by a storage control device of a file system according to an embodiment. [Figure 14] 10 is a flowchart illustrating a part of the flow of processing executed by a storage control device of a file system according to an embodiment. [Figure 15] 10 is a flowchart illustrating a part of the flow of processing executed by a storage control device of a file system according to an embodiment. [Figure 16] 10 is a flowchart illustrating a part of the flow of processing executed by a storage control device of a file system according to an embodiment. [Figure 17] FIG. 1 is a diagram illustrating an example of a configuration of a file system according to an embodiment. [Figure 18] FIG. 10 is a diagram illustrating another example of the configuration of a file system according to an embodiment. [Figure 19] FIG. 10 is a diagram illustrating another example of the configuration of a file system according to an embodiment. [Figure 20] FIG. 10 is a diagram illustrating another example of the configuration of a file system according to an embodiment. [Figure 21] FIG. 2 is a diagram illustrating an example of a hardware configuration of a storage control device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] There are two implementation methods for deduplication in storage devices: a file-based method, which performs deduplication on a file-by-file basis, and a block-based method, which performs deduplication on a fixed-length or variable-length block-by-block basis. This disclosure focuses on the file-based deduplication method.

[0015] [Embodiment 1] A first embodiment of the present disclosure will be described with reference to FIGS.

[0016] (file system) Fig. 1 shows an example of an implementation of a file system according to one embodiment. The storage device 2 shown in Fig. 1 differs from a storage device according to a related technology (not shown) in that an update-only hash table 313 is added to a disk 31. The storage control device 23 is an example of a storage control device.

[0017] 1, the storage control device 23 is connected to a plurality of host computers 111 to 114. Data read or data write requests from the host computers 111 to 114 are sent to the storage control device 23 via the host interface 21 of the storage device 2. The storage control device 23 accesses the disks 31 via the disk interface 22 based on the requests from the host computers 111 to 114.

[0018] 1, the disk 31 is shown as a single physical disk, but it may also be implemented as a virtual disk that combines multiple disks. In addition to using a magnetic disk as the physical disk, a nonvolatile semiconductor memory such as an SSD (Solid State Drive) may also be used.

[0019] 1, a hash table 312 used in deduplication processing is stored in the disk 31 in addition to a directory 311 and actual data 3151 and 3152. Note that the actual data 3152 and subsequent data (e.g., 3153, 3154, etc.) are also recorded on the disk 31.

[0020] (Storage control device 23) FIG. 2 is a block diagram showing an example of the configuration of the storage control device 23 in the storage device 2. As shown in FIG.

[0021] As shown in FIG. 2, the storage control device 23 is equipped with functions such as a file system 231, a RAID engine 232, deduplication 233, data compression 234, and thin provisioning 235.

[0022] Although the file system 231 and deduplication 233 are essential, other functions may or may not be implemented. Each function may be realized by software or hardware. Furthermore, the storage control device 23 may also be implemented with functions other than those described above.

[0023] (Directory 311) Fig. 3 shows an example of the structure of the directory 311. In Fig. 3, file information 3111 and 3112 are recorded in the directory 311.

[0024] The file information 3111 includes an identifier 31111 for identifying the file, a logical address 31112 indicating the recording location of the file within the storage device 2, a physical address 31113 indicating the actual data storage location, a main hash value 31114 used in deduplication processing on a file-by-file basis, and control information 31115 used in other file control. Additionally, the file information 3111 includes an additional storage field for a secondary hash value 31116 having a lower bit length than the primary hash value 31114.

[0025] The file information 3112 contains an identifier 31121 for identifying the file, a logical address 31122 indicating the recording location of the file in the storage device 2, a physical address 31123 indicating the actual data storage location, a primary hash value 31124 used in deduplication processing on a file-by-file basis, and control information 31125 used in other file control. In addition, the file information 3112 also contains an additional field for storing a secondary hash value 31126.

[0026] In the directory 311, file information 3112 and subsequent information (for example, 3113, 3114, etc.) is also recorded on the disk 31. The directory 311 may have a tree structure that incorporates other directories 311 therein, and the file system 231 of the storage control device 23 also supports the tree structure of the directory 311.

[0027] (Structure of hash table 312) 4 shows the structure of the hash table 312. The hash table 312 includes a hash entry search database 3121 and hash entries 31221, 31222, and 31223.

[0028] The hash entry search database 3121 stores information for searching for the hash entry 31221 of file A from the main hash value 312211 of file A. Generally, the hash entry search database 3121 has a tree structure.

[0029] The hash entry 31221 stores the main hash value 312211, the number of duplicates 312212, the physical address 312213, and the control information 312214 of the file A. The hash entry 31221 may also store a pointer to the hash value to be searched next.

[0030] (Hash entry search database 3121) FIG. 5 shows an example of the implementation of the hash entry search database 3121.

[0031] 5, when searching for hash entries 31221, 31222, and 31223, the search first begins with the hash table root 31211. The main hash value to be searched is compared with the hash values ​​in the hash value range p, q, ... of the hash table root 31211. Next, the upper branch 312121 linked to the matched hash value range p is searched.

[0032] Next, a search is made for lower branches associated with the hash value range pp, qq, . . . of the upper branch 312121.

[0033] In one example, the hash value ranges pp, qq, . . . of the upper branch 312121 and If a match is found in the hash value range pp in comparison with the main hash value to be searched, then the lower branch 312131 is searched.

[0034] In one example, when a match is found in the hash value range ppp of the lower branch 312131, the physical address 313114 is referenced and the hash entries 31221, 31222, 31223, etc. are compared with the main hash value being searched for. If a match is found, deduplication processing is performed based on the information in that hash entry. If a match is not found, a new hash entry is added.

[0035] The hash entry search database 3121 also records upper branches from the upper branch 312122 onwards (for example, 312123, 312124, etc.) and lower branches from the lower branch 312131 onwards (for example, 312132, 312133, etc.).

[0036] (Update-only hash table 313) Fig. 6 shows an example of the update-only hash table 313. In Fig. 6, secondary hash entries 3131 to 3133 are implemented in the update-only hash table 313. Each secondary hash entry 3131 is stored at a relative address arithmetically determined from secondary hash values ​​31116 and 31126 (Fig. 1) stored in the directory 311. A plurality of sub-entries 31311, 31312, and 31313 are stored in the secondary hash entry 3131.

[0037] The subentry 31311 stores the main hash value (new) 313111 of the updated file A', the main hash value (old) 313112 of the file A before the update, the hash entry address 313113 of the updated file A', the physical address 313114 of the updated file A', and control information 313115.

[0038] Since the secondary hash values ​​used in the update-only hash table 313 have a low bit length, there may be cases where the secondary hash values ​​match but the data do not match.

[0039] In general deduplication processing, a situation where the hash values ​​match but the data is different is called a collision. Anticipating the occurrence of such collisions, the update-only hash table 313 provides multiple sub-entries for each secondary hash entry. For example, in Figure 6, three sub-entries 31311, 31312, and 31313 are implemented within secondary hash entry 3131.

[0040] The number of sub-entries to be prepared in the secondary hash entry 3131 varies depending on the implementation of the storage device 2.

[0041] (Increased capacity of update-only hash table 313) With reference to FIG. 7, a method for increasing the capacity of a secondary hash entry when there are many different files with the same secondary hash value and the sub-entries 31311 in the secondary hash entry 3131 are exhausted will be described.

[0042] First, the contents of the subentry 31311 in the secondary hash entry 3131 are updated. The contents written in the subentry 31311 are saved in advance, and the main hash value (new) 313111, main hash value (old) 313112, and hash entry address 313113 in the subentry 31311 are all invalidated, and the physical address 314114 of the added secondary hash entry 3141 is stored in the physical address 313114.

[0043] Finally, the control information 313115 is rewritten (entry addition), and a link from the sub-entry 31311 to the added secondary hash entry 3141 is maintained.

[0044] The additional secondary hash entry 3141 secures an additional area 314 in the update-only hash table 313. In the first subentry 31411 of the additional secondary hash entry 3141, information of the subentry 31311 saved in advance is copied.

[0045] In the expansion area 314 of the update-only hash table 313, the sub-entries subsequent to the expansion secondary hash entry 3141 are also recorded on the disk 31.

[0046] (Update procedure for update-only hash table 313) The procedure for changing the directory 311 and the update-only hash table 313 when performing data update processing will be described with reference to FIGS.

[0047] (Directory 311 before file A is updated) 8, it is assumed that a process to update file A to file A' has occurred in the VDI environments of users A to D. Before the data update, file information about file A of each user is stored as file information 3111, 3112, 3113, and 3114 in directory 311 as a different file for each user.

[0048] 8, before the file information 3111 to 3114 is updated, the pre-update secondary hash value A is stored in the secondary hash value 31116. The relative address of the secondary hash entry 3131 is arithmetically determined from the pre-update secondary hash value A.

[0049] In the file information 3111 before the update, the physical address A of the file A before the update is stored in the physical address 31113. By comparing the physical address A stored in the physical address 313114 of the sub-entry 31311 in the secondary hash entry 3131 with the physical address A of the file A before the update, it is confirmed that the main hash value (new) 313111 of the sub-entry 31311 matches the data of the file A before the update.

[0050] In the file system according to the present disclosure, deduplication processing is performed, so a sub-entry that matches the physical address A of the file A before the update always exists in the secondary hash entry 3131. If no such sub-entry exists, it is determined that the directory 311 or the secondary hash entry 3131 is corrupted, and serious fault processing (shutting down the file system) is performed.

[0051] Since file A is common to users A to D, it is written in directory 311 so as to refer to the entity of file A at the same physical address A.

[0052] For example, the physical address A of file A is registered in the physical address 31113 of the file information 3111, and the main hash value A, which is the hash value of file A, is registered in the main hash value 31114.

[0053] The secondary hash value 31116 stores the data of the secondary hash value A calculated using the algorithm for the update-only hash table 313. For other file information, the actual data is the same, so the same primary hash value A, the same physical address A, and the same secondary hash value A are stored.

[0054] 9 shows the update process of the directory 311 and the update-only hash table 313 when user A updates file A to file A'. In FIG. 9, when user A updates file A to file A', the updated file A' is written from the host computer 111 to the storage device 2.

[0055] The storage control device 23 calculates the secondary hash value A' from the written file A', reads the secondary hash value 31116 of the file information 3111, and compares it with the secondary hash value A' calculated from file A'. If the comparison results in a mismatch, it determines that this is a file update process, and reads from the update-only hash table 313 the secondary hash entry 3132 that corresponds to the secondary hash value A' calculated from file A'.

[0056] 9, the subentry in the secondary hash entry 3132 is "empty," so a new subentry 31321 is created. A new primary hash value A' is calculated from the updated file A', and the hash entry A' and physical address A' are obtained from the hash table 312 (FIG. 4).

[0057] The acquired hash entry A′ and physical address A′ are then stored in the hash entry address 313213 and physical address 313214 of the secondary hash entry 3132 .

[0058] The calculated main hash value A' is registered in main hash value (new) 313211, and hash value A is read from main hash value 31114 that was written in file information 3111 before the update and registered in main hash value (old) 313212. Finally, control information 313215 of the new subentry 31321 is updated to indicate that the information in subentry 31321 is valid.

[0059] Although FIG. 9 does not show the update process of the secondary hash entry 3131 corresponding to the file A before the update, the update process of the secondary hash entry 3131 will be explained later with reference to FIG.

[0060] After the update process for the secondary hash entry 3132 (FIG. 9) and the secondary hash entry 3131 (FIG. 11) is completed, the file information 3111 corresponding to the identifier A of the host computer 111 is updated.

[0061] 9, the contents of the physical address 31113, main hash value 31114, and secondary hash value 31116 are updated from physical address A to physical address A', from hash value A to hash value A', and from secondary hash value A to secondary hash value A', respectively. If necessary, the control information 31115 is also updated.

[0062] Subsequently, when user B updates file A to file A′, file A′, which is an update of file A, is written to storage device 2.

[0063] The storage control device 23 obtains the secondary hash value A' from the written file A', and also reads the secondary hash value 31126 of the file information 3112 corresponding to the identifier B of the host computer 112, and compares it with the secondary hash value A' of the written file A'. If the comparison results in a mismatch, it determines that this is a data update process, and reads the secondary hash entry 3132 corresponding to the secondary hash value A' from the update-only hash table 313 (FIG. 6).

[0064] 10, sub-entry 31321, which was created when user A updated file A to file A', already exists in secondary hash entry 3132. The hash value A of main hash value (old) 313212 in sub-entry 31321 is compared with the main hash value A before the update in file information 3112, and if it is confirmed that the main hash values ​​before the update match, it is determined that file A has been updated to file A', and the actual data after the update is compared.

[0065] The file A' transferred from the host computer 112 to the storage device 2 is compared with the actual data stored in the area pointed to by the physical address 313214 in the sub-entry 31321 of the secondary hash entry 3132 to confirm that the data match.

[0066] If the data matches, the file information 3112 in the directory 311 and the number of duplicates 312212 of the hash entry 31221 are updated using the hash entry address A' stored in the hash entry address 313213 in the subentry 31321 without using the hash entry search database 3121 (Figure 4).

[0067] The update process for the secondary hash entry 3131 corresponding to the file A before the update will be described with reference to FIG.

[0068] After the update process for the secondary hash entry 3132 and the secondary hash entry 3131 is completed, the file information 3112 corresponding to the identifier B of the host computer 112 is updated.

[0069] 10, the contents of the physical address 31123, the main hash value 31124, and the sub hash value 31126 of the file information 3112 are updated. If necessary, the control information 31125 is also updated.

[0070] (Update process for secondary hash entry 3131) 11, the process of updating the secondary hash entry 3131 corresponding to the pre-update file A will be described. As described above, when updating the file information 3111 or the file information 3112, the secondary hash entry 3131 is identified from the pre-update secondary hash value 31116 or the pre-update secondary hash value 31126.

[0071] Next, the sub-entry 31311 of the secondary hash entry 3131 is identified from the pre-update physical address 31113 or physical address 31123. Then, the hash entry 31221 (FIG. 4) to be updated is identified in the hash table 312 (FIG. 4) from the hash entry address 313113 of the identified sub-entry 31311.

[0072] In the processing shown in FIGS. 9 and 10, file A is replaced with file A', and therefore the number of users who refer to the actual data of file A decreases.

[0073] The number of duplicates 312212 is managed in hash entry 31221 of hash table 312 shown in Fig. 4. Each time the number of users who refer to file A before update decreases, the number of duplicates 312212 included in hash entry 31221 of file A is decremented by one from hash table 312 shown in Fig. 4.

[0074] (Invalidating subentry 31311) 11 shows the state after users A to D have completed the update process from file A to file A'. All physical addresses 31113 of file information 3111 to 3114 have been updated to physical addresses A'. The update process from file A to A' means that the number of users referencing file A will decrease. At this time, the number of duplicates 312212 included in hash entry 31221 of file A is zero (none).

[0075] When the number of duplicates 312212 of the hash entry 31221 becomes 0, file A is no longer referenced anywhere, so the hash entry is deleted and the area for the actual data of file A is released.

[0076] Since the secondary hash entry 3131 is referenced as a trigger for the deletion process of the hash entry 31221, when it is detected that the number of duplicates of the hash entry 31221 has become 0, the invalidation process of the secondary hash entry 3131 can be executed immediately.

[0077] After the number of duplicates 312212 of the hash entry 31221 becomes 0 and the hash entry 31221 is deleted, the sub-entry 31311 of the secondary hash entry 3131, which references the same physical address A, is invalidated. Then, information indicating entry invalidation is written to the control information 313115 of the sub-entry 31311.

[0078] (Data update process) The flow of the data update process will be described with reference to Fig. 12 to Fig. 16. Note that in data update processes other than the first time, the flow follows the order of Fig. 12, connector A, Fig. 13, connector D, Fig. 14, connector G, and Fig. 16.

[0079] (Data writing process) The data write process will be described with reference to FIG.

[0080] 12, when the storage device 2 receives a data write request from one of the host computers 111 to 114, it calculates a secondary hash value from the received data (S101). The secondary hash value may be calculated using a dedicated algorithm, or may be substituted with an existing check code such as CRC-16.

[0081] When using an existing check code, it is expected that the calculation result will be a specified value (usually 0) when all data including the check code is received. Therefore, the code attached as the check code itself is used as the secondary hash value.

[0082] The directory 311 is read, and the identifier or logical address of the write data is compared with the identifier or logical address of the file information in the directory 311 (S102). Then, depending on whether or not there is file information that matches the data, it is determined whether it is a new write or an update process for an existing file (S103).

[0083] Taking the configuration of directory 311 in Fig. 3 described above as an example, identifier 31111 of file information 3111 is compared with the identifier of the written data, and if they match (Yes in S103), it is determined that the process is to update file information 3111. On the other hand, if they do not match (No in S103), identifier 31121 of the next file information 3112 is compared with the identifier of the written data (S105). Thereafter, file information in directory 311 is compared sequentially with the identifier of the written data.

[0084] If it is determined that the data write is an update process for an existing file (Yes in S103), the secondary hash value calculated when the data was received is compared with the secondary hash value recorded in the file information 3111, and a primary determination is made as to whether the data write is an overwrite of the same data (S104).

[0085] 3 as an example, if the identifier 31111 of the file information 3111 matches the identifier of the written data (Yes in S103), the secondary hash value of the written data is compared with the secondary hash value 31116 of the file information 3111 (S104). If they do not match (No in S104), the process proceeds to update the update-only hash table 313 (connector A). If they match (Yes in S104), the written data is compared with the actual data (S105).

[0086] Physical address A is obtained from physical address 31113 of file information 3111, actual data is read from physical address A, and a comparison is made between the actual data and the written data (S105). If the data comparison results in a match (Yes in S106), the same data is to be overwritten and saved, so update processing such as update history is performed on control information 31115 of file information 3111, and directory 311 is updated (S107). However, because the actual data and the written data match, no change to the actual data and no update processing for deduplication are performed.

[0087] (Update process for update-only hash table 313) The flow of the search and update process of the update-only hash table 313, which is newly implemented in this disclosure, will be described with reference to Fig. 13. Note that explanations of steps other than those numbered in Fig. 13 will be omitted.

[0088] 13, in both the case of updating an existing file (branching from connector A in FIG. 12) and the case of creating a new file (branching from connector B in FIG. 12), the update-only hash table 313 is read and the secondary hash entry 3131 for the secondary hash value of the write data is searched for (S201). Here, a description of the branch starting from connector B is omitted.

[0089] If the subentry in the secondary hash entry 3131 that corresponds to the secondary hash value of the written file is "empty" (no valid subentry exists) (Yes in S202), a new subentry creation process is performed (S204). If the subentry in the secondary hash entry 3131 is not "empty" (no valid subentry exists) (No in S202), it is determined that a file update request had previously been made from another user's VDI environment, and that update processing had been performed using the secondary hash entry, and it is determined whether the primary hash value (old) matches the primary hash value in the directory before the update (S203). By comparing the primary hash value (old) with the primary hash value in the directory before the update, the number of actual data comparisons can be reduced if a collision occurs in the secondary hash entry 3131. Explanation of some of the flow from step S203 onwards will be omitted.

[0090] 13, if all valid subentries in the secondary hash entry 3131 have been searched and no data matches (Yes in S205), a subentry addition process is performed (S206) (connector C). Also, if a subentry needs to be added and the secondary hash entry 3131 is already "full" (Yes in S207), the process of adding an additional secondary hash entry 3141 is performed (connector F), as explained in Fig. 7. Note that a description of the flow beyond connector F, i.e., the process of adding an additional secondary hash entry 3141, will be omitted.

[0091] If matching data is found, the process increases the number of duplicates in the hash entry by 1 (S208), and then proceeds to the flow of hash entry update processing for pre-update data shown in FIG. 14 (connector D).

[0092] (Update process of hash entry of data before update) The flow of hash entry update processing of pre-update data will be described with reference to FIG.

[0093] First, the previous sub-hash value and the previous physical address are extracted from the file information before updating (S301).

[0094] Next, the update-only hash table 313 is referenced to identify a secondary hash entry on the update-only hash table 313 (S302). Then, the physical address on the sub-entry in the identified secondary hash entry is referenced and compared with the previous physical address (S303).

[0095] If the previous physical addresses match (Yes in S303), the hash entry is identified from the hash entry address in the subentry, and the number of duplicate hash entries is decremented by 1 (S304).Then, the flow proceeds to the new data write process shown in Figure 16 (connector G).

[0096] When searching for a subentry of the secondary hash entry, if a subentry matching the previous physical address is not found (Yes in S305), this means that the file registered in the directory 311 does not exist, and it is determined to be a serious failure. In this case, the probability of other data cannot be guaranteed, so the service of the storage device 2 should be stopped.

[0097] (New data writing process) FIG. 15 shows the flow of the new data writing process continuing from the connector E in FIG.

[0098] 15, in the process of writing new data, first, a primary hash value is calculated (S401), and then the deduplication hash table is searched (S402) to determine whether there is an entry with a matching primary hash value (S403).

[0099] If there is no entry with a matching main hash value (No in S403), a process for adding a hash entry is executed (S404), and the process proceeds to the data write end process flow (connector H) shown in FIG.

[0100] On the other hand, if there is an entry with a matching primary hash value (Yes in S403), the write data is compared with the actual data (S405). If the write data and the actual data do not match (No in S406), collision processing is performed. A detailed description of collision processing will be omitted in this disclosure.

[0101] On the other hand, if the write data and the actual data match (Yes in S406), the hash entry is updated (+1) (S407), and the flow then proceeds to the data write termination processing flow (connector I) shown in FIG.

[0102] (Data writing completion process) The flow of data writing termination processing will be described with reference to Fig. 16. As shown in Fig. 16, in the branch starting from connector G, processing is performed when previously updated data is updated in the VDI environment of another user.

[0103] If the hash entry of the pre-update data has not been deleted (No in S501), the update-only hash table 313 itself does not perform any particular update processing, and only updates the directory 311 (S502).

[0104] On the other hand, if the hash entry of the pre-update data has been deleted (Yes in S501), the update-only table is read and the secondary hash entry is referenced (S503), and the sub-entry of the secondary hash entry in the update-only hash table 313 is deleted (S504).

[0105] In the branch starting from the connector H, new data creation processing is performed. First, the update-only hash table 313 is read and the secondary hash entry is referenced (S505). New data registration processing is performed in the sub-entry of the secondary hash entry in the update-only hash table 313. To register the new data, the main hash value (new), hash entry address, and physical address are registered. The main hash value (old) is invalidated (S506). Then, the directory 311 is updated (S502).

[0106] 16, the branch starting from connector I represents the initial flow of data update processing. First, the update-only hash table 313 is read and the secondary hash entry is referenced (S507). In the sub-entry of the secondary hash entry in the update-only hash table 313, the main hash value (new), main hash value (old), hash entry address, and physical address are registered (S508). Then, the directory 311 is updated (S502).

[0107] In a VDI environment or the like, when multiple users update the same file, it is possible to omit the process of searching the hash entry search database 3121 of the hash table 312 (which places the greatest load on the file in the deduplication process) and the process of calculating the main hash value of the file (which places the greatest load on the CPU) except for the first time.

[0108] Furthermore, since the deduplication process can be performed simply by adding or subtracting the hash entry duplicates, and the hash entries can be referenced from the update-only hash table 313, the deduplication process of the present disclosure can reduce the load of searching the hash entry search database compared to related technologies.

[0109] (Effects of this embodiment) According to the configuration of this embodiment, the secondary hash value calculated from the data file written to the storage device 2 is compared with the secondary hash value included in the file information stored in the directory 311. If the secondary hash value of the data file written to the storage device 2 does not match the secondary hash value included in the file information stored in the directory 311, the secondary hash entry corresponding to the secondary hash value of the data file written to the storage device 2 is read from the update-only hash table 313. The main hash value (old) of the data file before the update is compared with the main hash value before the update stored in the directory 311 using the sub-entry included in the secondary hash entry. If a match is detected between the main hash value (old) of the data file before the update and the main hash value stored in the directory 311, the data file written to the storage device 2 is compared with the latest data file. If a match is confirmed between the data file written to the storage device 2 and the latest data file, it is determined to be duplicate data, and a process of updating the file information is executed using the main hash value and hash entry address of the latest data file, and the physical address of the latest data file.

[0110] This makes it possible to omit the process of searching the hash entry search database 3121 of the hash table 312 and the process of calculating the main hash value of the file except for the first time, thereby reducing the calculation load on the file management system 1.

[0111] [Embodiment 2] A second embodiment of the present disclosure will be described with reference to Figures 17 to 19. In the second embodiment, components common to the first embodiment are denoted by the same reference numerals as in the first embodiment, and descriptions thereof will be omitted.

[0112] (File Management Systems 1A~1C) 17 shows an example of implementation of a file management system 1A according to one embodiment. In the file management system 1 of the first embodiment described above, the directory 311, hash table 312, update-only hash table 313, extension area 314 for the update-only hash table, and actual data 3151 and 3152 are all recorded on a single physical disk or virtual disk 32.

[0113] In the second embodiment, actual data 3251 and 3252 are recorded on a physical disk 31 and virtual disks 32, 33, . . .

[0114] In FIG. 17, the directory 311, hash table 312, update-only hash table 313, and expansion area 314 for update-only hash table 313 are recorded on a single physical disk 31 or virtual disk 32, but implementations in which these are recorded on separate physical disks or virtual disks are also possible.

[0115] In FIG. 17, physical disks or virtual disks 32 and 33 are shown, but the number of disks is not limited to three, and a fourth or subsequent disk may be installed.

[0116] Fig. 18 is an example of implementation of a file management system 1B according to an embodiment. Fig. 18 shows an example in which the update-only hash table 313 resides in the cache memory of the storage device 2 and is used as the on-cache update-only hash table 23313. Because the update-only hash table 313 has a small capacity, it can be made resident in the cache memory if the cache memory capacity of the storage device is large.

[0117] This allows update processing to be performed on-cache, making it possible to further increase speed.

[0118] Fig. 19 is an implementation example of a file management system 1C according to an embodiment. Fig. 19 shows an example in which a common physical or virtual disk 31 is accessed from a plurality of storage devices 2. In Fig. 19, host computers 111 to 114 are connected to the storage device 2 and the storage device 4. The storage device 2 and the storage device 4 share the disk 31.

[0119] When the storage device 2 and the storage device 4 execute the deduplication process, they access the directory 311, the hash table 312, the update-only hash table 313, and the actual data 3151, 3152, etc., respectively.

[0120] When data on a common disk is accessed from multiple storage devices, processes such as exclusive control are required to maintain data consistency, but these can be realized using well-known techniques.

[0121] 19, the disk 31 is shared by two storage devices, storage device 2 and storage device 4, but the disk 31 may be shared by three or more devices. Also, multiple disks 31 may be shared among multiple storage devices.

[0122] (Effects of this embodiment) According to the configuration of this embodiment, the secondary hash value calculated from the data file written to the storage devices 2 and 4 is compared with the secondary hash value included in the file information stored in the directory 311. If the secondary hash value of the data file written to the storage devices 2 and 4 does not match the secondary hash value included in the file information stored in the directory 311, the secondary hash entry corresponding to the secondary hash value of the data file written to the storage devices 2 and 4 is read from the update-only hash table 313. The main hash value (old) of the data file before the update is compared with the main hash value before the update stored in the directory 311 based on the sub-entry included in the secondary hash entry. If a match is detected between the main hash value (old) of the data file before the update and the main hash value stored in the directory 311, the data file written to the storage device 2 is compared with the latest data file. If a match is confirmed between the data file written to the storage device 2 and the latest data file, it is determined that the data is duplicate data, and a process of updating the file information is executed using the main hash value and hash entry address of the latest data file, and the physical address of the latest data file.

[0123] This makes it possible to omit the process of searching the hash entry search database 3121 of the hash table 312 and the process of calculating the main hash value of the file except for the first time, thereby reducing the computational load on the file management systems 1A, 1B, 1C, and 1D.

[0124] [Embodiment 3] A third embodiment of the present disclosure will be described with reference to Fig. 20. In the third embodiment, components common to the first and second embodiments are denoted by the same reference numerals as those in the first and second embodiments, and description thereof will be omitted.

[0125] (Server 5) 20 shows an example in which the deduplication processing program described in the first embodiment is installed in a server 5, and the server's internal disk is provided as the storage device 2 described in the first and second embodiments. As shown in FIG. 20, the server 5 reads a storage control program 3261 from the virtual disk 32, loads it into the memory in the server 5, and executes the above-mentioned deduplication processing as storage control (program) 53.

[0126] This makes it possible to reduce storage capacity by deduplicating common files when multiple users use the same or similar environment, such as in a VDI environment.

[0127] (Differences from related technologies) In related technology, when users update a common file, they have to perform deduplication processing individually.

[0128] On the other hand, with the configuration of this embodiment, when updating files shared by many users, such as during regular OS updates, it is possible to simplify the update process by focusing on the fact that the updated files are often also shared, thereby reducing the load of the deduplication process compared to related technologies.

[0129] Each component of the storage control device 23 described in the first to third embodiments is represented by a functional block. Some or all of these components are realized by an information processing device such as that shown in Fig. 21. Fig. 21 is a block diagram showing an example of the hardware configuration of the information processing device.

[0130] 21, the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other. Note that the computer 110 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to or instead of the CPU 111.

[0131] The CPU 111 loads the programs (codes) of this embodiment stored in the storage device 113 into the main memory 112 and executes them in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory). The programs of this embodiment are provided in a state stored in a computer-readable recording medium 120. The programs of this embodiment may be distributed over the Internet connected via the communication interface 117.

[0132] Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to a display device 119 and controls the display on the display device 119.

[0133] Data reader / writer 116 mediates data transmission between CPU 111 and recording medium 120, reads programs from recording medium 120, and writes processing results from computer 110 to recording medium 120. Communication interface 117 mediates data transmission between CPU 111 and other computers.

[0134] Specific examples of recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as flexible disks, or optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0135] (Addendum) Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0136] (Appendix 1) A storage control device that controls a storage device, the storage device is connected to a directory, a hash table, an update-only hash table, and a disk; The disk stores actual data files, the directory stores file information including a physical address of the actual data file, a main hash value used in the deduplication process of the actual data file, and a sub hash value having a bit length shorter than that of the main hash value; The hash table stores hash entries and a hash entry search database for searching for the hash entries from the main hash values, a secondary hash entry corresponding to the secondary hash value is stored in the update-only hash table; the secondary hash entry includes a main hash value (new) and a hash entry address of the latest data file, a main hash value (old) of the data file before the update, and a physical address of the latest data file; comparing a secondary hash value calculated from the data file written to the storage device with the secondary hash value included in the file information stored in the directory; If the secondary hash value of the data file written to the storage device does not match the secondary hash value included in the file information stored in the directory, The secondary hash entry corresponding to the secondary hash value of the data file written to the storage device is read from the update-only hash table, and the main hash value (old) of the data file before the update is compared with the main hash value before the update written in the directory based on the sub-entry included in the secondary hash entry. If a match between the main hash value (old) of the data file before the update and the main hash value written in the directory is detected, the data file written to the storage device is compared with the latest data file. If a match between the data file written to the storage device and the latest data file is confirmed, it is determined to be duplicate data, and a process of updating the file information is performed using the main hash value and hash entry address of the latest data file, and the physical address of the latest data file. Storage control device.

[0137] (Appendix 2) If the secondary hash value of the data file written to the storage device matches the secondary hash value included in the file information stored in the directory, comparing the data file written to the storage device with the data file saved in the storage device; Even if the data file written to the storage device does not match the data file saved in the storage device, Execute the process to update the file information 2. The storage control device according to claim 1,

[0138] (Appendix 3) Within the secondary hash entry, multiple sub-entries are implemented. 2. The storage control device according to claim 1,

[0139] (Appendix 4) If there is no sub-entry in the secondary hash entry that corresponds to the secondary hash value of the data file written to the storage device, A new sub-entry corresponding to the data file written to the storage device is created in the secondary hash entry. 2. The storage control device according to claim 1,

[0140] (Appendix 5) When the actual data file stored on the disk is updated by the data file written to the storage device, The secondary hash entry stores a new primary hash value corresponding to the updated data file and an old primary hash value corresponding to the actual data file before the update. 2. The storage control device according to claim 1,

[0141] (Appendix 6) the directory, the hash table, and the update-only hash table are stored on a physical disk; The actual data files are stored on one or more virtual disks. 2. The storage control device according to claim 1,

[0142] (Appendix 7) If the sub-entries in the secondary hash entry are full, an expansion area for expanding the sub-entries is secured in the update-only hash table. 2. The storage control device according to claim 1,

[0143] (Appendix 8) The hash entry includes the main hash value corresponding to the actual data file, a number of duplicates indicating the number of host computers that reference the actual data file, the physical address of the actual data file, and control information used in file control. 2. The storage control device according to claim 1,

[0144] (Appendix 9) A storage control method for controlling a storage device, comprising: the storage device is connected to a directory, a hash table, an update-only hash table, and a disk; The disk stores actual data files, the directory stores file information including a physical address of the actual data file, a main hash value used in the deduplication process of the actual data file, and a sub hash value having a bit length shorter than that of the main hash value; The hash table stores hash entries and a hash entry search database for searching for the hash entries from the main hash values, a secondary hash entry corresponding to the secondary hash value is stored in the update-only hash table; the secondary hash entry includes a main hash value (new) and a hash entry address of the latest data file, a main hash value (old) of the data file before the update, and a physical address of the latest data file; The storage control method includes: a process of comparing a secondary hash value calculated from the data file written in the storage device with the secondary hash value included in the file information stored in the directory; If the secondary hash value of the data file written to the storage device does not match the secondary hash value included in the file information stored in the directory, a process of reading from the update-only hash table the secondary hash entry corresponding to the secondary hash value of the data file written to the storage device, comparing the main hash value (old) of the data file before the update with the main hash value before the update recorded in the directory based on a sub-entry included in the secondary hash entry, and if a match is detected between the main hash value (old) of the data file before the update and the main hash value recorded in the directory, comparing the data file written to the storage device with the latest data file, and if a match is confirmed between the data file written to the storage device and the latest data file, determining that the data is duplicate data, and updating the file information using the main hash value and hash entry address of the latest data file and the physical address of the latest data file; A storage control method for causing a computer to execute the above.

[0145] (Appendix 10) A storage control program for controlling a storage device, the storage device is connected to a directory, a hash table, an update-only hash table, and a disk; The disk stores actual data files, the directory stores file information including a physical address of the actual data file, a main hash value used in the deduplication process of the actual data file, and a sub hash value having a bit length shorter than that of the main hash value; The hash table stores hash entries and a hash entry search database for searching for the hash entries from the main hash values, a secondary hash entry corresponding to the secondary hash value is stored in the update-only hash table; the secondary hash entry includes a main hash value (new) and a hash entry address of the latest data file, a main hash value (old) of the data file before the update, and a physical address of the latest data file; The storage control program a process of comparing a secondary hash value calculated from the data file written in the storage device with the secondary hash value included in the file information stored in the directory; If the secondary hash value of the data file written to the storage device does not match the secondary hash value included in the file information stored in the directory, a process of reading from the update-only hash table the secondary hash entry corresponding to the secondary hash value of the data file written to the storage device, comparing the main hash value (old) of the data file before the update with the main hash value before the update recorded in the directory based on a sub-entry included in the secondary hash entry, and if a match is detected between the main hash value (old) of the data file before the update and the main hash value recorded in the directory, comparing the data file written to the storage device with the latest data file, and if a match is confirmed between the data file written to the storage device and the latest data file, determining that the data is duplicate data, and updating the file information using the main hash value and hash entry address of the latest data file and the physical address of the latest data file; A storage control program that causes a computer to execute the above.

[0146] Furthermore, some or all of the configurations described in Supplementary Notes 2 to 8 that are dependent on Supplementary Note 1 (e.g., storage control device) may also be dependent on Supplementary Note 9 (e.g., storage control method) and Supplementary Note 10 (e.g., storage control program) in the same dependent relationship as Supplementary Notes 2 to 8. Furthermore, within the scope of each of the above-mentioned embodiments, some or all of the configurations described as Supplements may be made dependent on various hardware, software, various recording means for recording software, or systems.

[0147] The present disclosure has been described above with reference to several embodiments. However, the present disclosure is not limited to the above embodiments. Each embodiment can be combined with other embodiments as appropriate. Furthermore, various modifications that can be understood by those skilled in the art can be made to the configurations and details of the above embodiments within the scope of the present disclosure. [Industrial Applicability]

[0148] This disclosure assumes a case where a deduplication function is provided in a storage device connected to a server or cloud where many users use a virtual environment such as VDI. [Explanation of symbols]

[0149] 1. File Management System 1A File Management System 2. Storage devices 23 Storage control device 31 discs 111 host computer 112 host computer 113 Host Computer 114 Host Computer 311 Directory 312 Hash Table 313 Update-only hash table 314 Expansion Area 3121 Hash Entry Search Database 3131 Sub-Hash Entries 3132 Sub-Hash Entries 3133 Sub-Hash Entries 3141 Additional Sub-Hash Entry 31311 subentries 31312 subentries 31313 subentries

Claims

1. A storage control device that controls a storage device, the storage device is connected to a directory, a hash table, an update-only hash table, and a disk; The disk stores actual data files, the directory stores file information including a physical address of the actual data file, a main hash value used in the deduplication process of the actual data file, and a sub hash value having a bit length shorter than that of the main hash value; The hash table stores hash entries and a hash entry search database for searching for the hash entries from the main hash values, a secondary hash entry corresponding to the secondary hash value is stored in the update-only hash table; the secondary hash entry includes a main hash value (new) and a hash entry address of the latest data file, a main hash value (old) of the data file before updating, and a physical address of the latest data file; comparing a secondary hash value calculated from the data file written to the storage device with the secondary hash value included in the file information stored in the directory; If the secondary hash value of the data file written to the storage device does not match the secondary hash value included in the file information stored in the directory, The secondary hash entry corresponding to the secondary hash value of the data file written to the storage device is read from the update-only hash table, and the main hash value (old) of the data file before the update is compared with the main hash value before the update written in the directory based on the sub-entry included in the secondary hash entry. If a match between the main hash value (old) of the data file before the update and the main hash value written in the directory is detected, the data file written to the storage device is compared with the latest data file. If a match between the data file written to the storage device and the latest data file is confirmed, it is determined to be duplicate data, and a process of updating the file information is performed using the main hash value and hash entry address of the latest data file, and the physical address of the latest data file. Storage control device.

2. If the secondary hash value of the data file written to the storage device matches the secondary hash value included in the file information stored in the directory, comparing the data file written to the storage device with the data file saved in the storage device; Even if the data file written to the storage device does not match the data file saved in the storage device, Execute the process to update the file information The storage control device according to claim 1 .

3. Within the secondary hash entry, multiple sub-entries are implemented. The storage control device according to claim 1 .

4. If there is no sub-entry in the secondary hash entry that corresponds to the secondary hash value of the data file written to the storage device, A new sub-entry corresponding to the data file written to the storage device is created in the secondary hash entry. The storage control device according to claim 1 .

5. When the actual data file stored on the disk is updated by the data file written to the storage device, The secondary hash entry stores a new primary hash value corresponding to the updated data file and an old primary hash value corresponding to the actual data file before the update. The storage control device according to claim 1 .

6. the directory, the hash table, and the update-only hash table are stored on a physical disk; The actual data files are stored on one or more virtual disks. The storage control device according to claim 1 .

7. If the sub-entries in the secondary hash entry are full, an expansion area for expanding the sub-entries is secured in the update-only hash table. The storage control device according to claim 1 .

8. The hash entry includes the main hash value corresponding to the actual data file, a number of duplicates indicating the number of host computers that reference the actual data file, the physical address of the actual data file, and control information used in file control. The storage control device according to claim 1 .

9. A storage control method for controlling a storage device, comprising: the storage device is connected to a directory, a hash table, an update-only hash table, and a disk; The disk stores actual data files, the directory stores file information including a physical address of the actual data file, a main hash value used in the deduplication process of the actual data file, and a sub hash value having a bit length shorter than that of the main hash value; The hash table stores hash entries and a hash entry search database for searching for the hash entries from the main hash values, a secondary hash entry corresponding to the secondary hash value is stored in the update-only hash table; the secondary hash entry includes a main hash value (new) and a hash entry address of the latest data file, a main hash value (old) of the data file before updating, and a physical address of the latest data file; The storage control method includes: a process of comparing a secondary hash value calculated from the data file written in the storage device with the secondary hash value included in the file information stored in the directory; If the secondary hash value of the data file written to the storage device does not match the secondary hash value included in the file information stored in the directory, a process of reading from the update-only hash table the secondary hash entry corresponding to the secondary hash value of the data file written to the storage device, comparing the main hash value (old) of the data file before the update with the main hash value before the update written in the directory based on a sub-entry included in the secondary hash entry, and if a match between the main hash value (old) of the data file before the update and the main hash value written in the directory is detected, comparing the data file written to the storage device with the latest data file, and if a match between the data file written to the storage device and the latest data file is confirmed, determining that the data is duplicate data, and updating the file information using the main hash value and hash entry address of the latest data file and the physical address of the latest data file; A storage control method for causing a computer to execute the above.

10. A storage control program for controlling a storage device, the storage device is connected to a directory, a hash table, an update-only hash table, and a disk; The disk stores actual data files, the directory stores file information including a physical address of the actual data file, a main hash value used in the deduplication process of the actual data file, and a sub hash value having a bit length shorter than that of the main hash value; The hash table stores hash entries and a hash entry search database for searching for the hash entries from the main hash values, a secondary hash entry corresponding to the secondary hash value is stored in the update-only hash table; the secondary hash entry includes a main hash value (new) and a hash entry address of the latest data file, a main hash value (old) of the data file before updating, and a physical address of the latest data file; The storage control program a process of comparing a secondary hash value calculated from the data file written in the storage device with the secondary hash value included in the file information stored in the directory; If the secondary hash value of the data file written to the storage device does not match the secondary hash value included in the file information stored in the directory, a process of reading from the update-only hash table the secondary hash entry corresponding to the secondary hash value of the data file written to the storage device, comparing the main hash value (old) of the data file before the update with the main hash value before the update written in the directory based on a sub-entry included in the secondary hash entry, and if a match between the main hash value (old) of the data file before the update and the main hash value written in the directory is detected, comparing the data file written to the storage device with the latest data file, and if a match between the data file written to the storage device and the latest data file is confirmed, determining that the data is duplicate data, and updating the file information using the main hash value and hash entry address of the latest data file and the physical address of the latest data file; A storage control program that causes a computer to execute the above.

Images