Storage control device, storage control method, and program

Abstract

To reduce wasteful consumption of resources in managing deduplication in storage. [Solution] The storage control device 20 comprises a file processing unit 21 that reads or writes files in the storage area of ​​the storage in response to instructions from the host that uses the storage, a deduplication unit 22 that performs deduplication processing on the files that are the subject of reading or writing so that there are no duplicates in the storage area, and a duplication permission unit 23 that identifies the access status to the files in the storage area and, if the identified access status satisfies a set condition, excludes the corresponding file from the deduplication processing.

Description

[Technical Field]

[0001] The present disclosure relates to a storage control device and a storage control method that implement a deduplication function, and further to a program for realizing these. [Background technology]

[0002] VDI (Virtual Desktop Infrastructure) is a technology that creates multiple virtual PCs (Personal Computers) on a server. In a VDI environment, multiple users can use a common operating system (OS) and common applications on the virtual PCs, allowing for centralization of the OS and applications.

[0003] In an OS update in a VDI environment, an operation for updating is performed for each user on a common OS image. However, if the OS image before the update in each user's VDI environment is common, the OS image after the update will also be common among users.

[0004] Furthermore, storage devices that store VDI environments may be equipped with a deduplication function (see, for example, Patent Document 1). A system equipped with a deduplication function can process each user's OS and application as duplicate data, thereby improving the efficiency of storage capacity and speeding up read processing. Patent Document 1 also discloses a system that analyzes file access patterns to optimize the placement of deduplicated actual data in a storage device.

[0005] There are two implementation methods for deduplication: one that performs deduplication on a file-by-file basis, and one that performs deduplication on a data block-by-data block basis. An example of a file-by-file deduplication method is described below.

[0006] Generally, the development of programs such as application programs is divided among multiple people, and the use of file management tools such as SubVersion (hereafter referred to as "SVN") and Git improves the efficiency of the division of labor among multiple people.

[0007] For example, in the above-mentioned SVN, the program body is stored on the management server, so each developer places a local copy of the program body in their own VDI environment. This local copy will be identical to the program body unless modifications or other changes are made, so it is subject to deduplication. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Publication No. 2024-125451 Summary of the Invention [Problem to be solved by the invention]

[0009] However, if each user modifies or alters a file during the course of program development, the modified file will be excluded from the deduplication target because it will be different from files in other VDI environments.

[0010] Specifically, suppose a VDI environment is built on a storage device with deduplication enabled, and a file management tool is applied to this VDI environment and a development management server. In this case, when a user updates a file for program development, the updated file is different from the file of another user, so the deduplication function excludes it from deduplication and manages it as a newly created file.

[0011] In this way, even if a file that has been excluded from deduplication has barely been modified or altered, it will become an independent file that will not be shared by users other than the user who updated the file until the development management server updates the original file. As a result, there may be cases where management resources for the deduplication function in the storage device are wasted.

[0012] An example of an objective of the present disclosure is to suppress wasteful consumption of resources in managing deduplication in storage. [Means for solving the problem]

[0013] In order to achieve the above object, a storage control device according to one aspect of the present disclosure comprises: a file processing unit that reads or writes files in a storage area of ​​the storage in response to an instruction from a host that uses the storage; a deduplication unit that performs deduplication processing on the file to be read or written so that the file does not exist duplicated in the storage area; a duplication permission unit that identifies an access status of the file in the storage area, and, if the identified access status satisfies a set condition, excludes the corresponding file from the target of the deduplication process; The present invention is characterized in that it is provided with:

[0014] In order to achieve the above object, a storage control method according to one aspect of the present disclosure includes: a file processing step of reading or writing a file in a storage area of ​​the storage in response to an instruction from a host that uses the storage; a deduplication step of performing a deduplication process on the file to be read or written so that the file does not exist duplicated in the storage area; a duplication allowing step of identifying an access status of the file in the storage area, and excluding the corresponding file from the target of the deduplication process if the identified access status satisfies a set condition; The present invention is characterized by having the following:

[0015] Furthermore, in order to achieve the above object, a program according to one aspect of the present disclosure includes: On the computer, a file processing step of reading or writing a file in a storage area of ​​the storage in response to an instruction from a host that uses the storage; a deduplication step of performing a deduplication process on the file to be read or written so that the file does not exist duplicated in the storage area; a duplication allowing step of identifying an access status of the file in the storage area, and excluding the corresponding file from the target of the deduplication process if the identified access status satisfies a set condition; The method is characterized in that: [Effects of the Invention]

[0016] As described above, according to the present disclosure, it is possible to suppress unnecessary consumption of resources in managing deduplication in storage. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a storage control device. [Figure 2] FIG. 2 is a configuration diagram showing a more specific configuration of an example of a storage control device. [Figure 3] FIG. 3 is a diagram showing an example of a directory managed by a file system. [Figure 4] FIG. 4 is a diagram showing an example of the contents of the access information shown in FIG. [Figure 5]FIG. 5 is a diagram showing an example of a hash table for managing files to be de-duped when the de-duplication function is enabled. [Figure 6] FIG. 6 is a diagram showing an example of implementation of the hash entry search database shown in FIG. [Figure 7] FIG. 7 is a diagram illustrating an example of a VDI environment. [Figure 8] FIG. 8 is a diagram illustrating an example of the operation of a host when a file is locally copied in a VDI environment. [Figure 9] FIG. 9 is a diagram illustrating an example of the operation of a host when a file is locally updated in a VDI environment. [Figure 10] FIG. 10 is a diagram showing operations subsequent to the operations shown in FIG. [Figure 11] FIG. 11 is a diagram showing operations subsequent to the operations shown in FIG. [Figure 12] FIG. 12 is a diagram illustrating an example of the operation of a host when a local update of a file is canceled in a VDI environment. [Figure 13] FIG. 13 is a diagram illustrating an example of the operation of a host when local file updates are repeatedly performed in a VDI environment. [Figure 14] FIG. 14 is a flow diagram illustrating an example of the operation of the storage control device. [Figure 15] FIG. 15 is a flow diagram illustrating an example of the operation of the storage control device. [Figure 16] FIG. 16 is a flow diagram illustrating an example of the operation of the storage control device. [Figure 17] FIG. 17 is a flow diagram illustrating an example of the operation of the storage control device. [Figure 18] FIG. 18 is a diagram showing an example of the content of processing for each operation mode when deduplication is performed. [Figure 19] FIG. 19 is a diagram showing an example of the content of processing for each operation mode when deduplication is performed. [Figure 20]FIG. 20 is a diagram showing an example of the content of processing for each operation mode when deduplication is performed. [Figure 21] FIG. 21 is a diagram showing an example of the content of processing for each operation mode when deduplication is performed. [Figure 22] FIG. 22 is a diagram showing an example of the content of processing for each operation mode when deduplication is performed. [Figure 23] FIG. 23 is a diagram showing a list of notations for the execution times of the processes shown in FIGS. [Figure 24] FIG. 24 is a diagram showing a list of definitions of the execution times of the processes shown in FIGS. [Figure 25] FIG. 25 is a diagram showing definitions of parameters that indicate the access frequency of each data write pattern when deduplication is enabled. [Figure 26] FIG. 26 is a diagram showing an example of a formula for calculating a performance index using the write load performance index shown in FIG. 24 and the access frequency parameter shown in FIG. [Figure 27] FIG. 27 is a diagram illustrating an example of a formula for calculating the deduplication performance difference. [Figure 28] FIG. 28 is a diagram illustrating another example of the formula for calculating the deduplication performance difference. [Figure 29] FIG. 29 is a block diagram illustrating an example of a computer that realizes a storage control device. DETAILED DESCRIPTION OF THE INVENTION

[0018] (Embodiment) Hereinafter, a storage control device, a storage control method, and a program according to an embodiment will be described with reference to FIGS.

[0019] [Device configuration] First, the schematic configuration of an example of a storage control device will be described with reference to Fig. 1. Fig. 1 is a configuration diagram showing the schematic configuration of a storage control device.

[0020] 1 is a device for controlling storage (storage device). As shown in FIG. 1, the storage control device 20 includes a file processing unit 21, a deduplication unit 22, and a duplication permission unit 23.

[0021] Of these, the file processing unit 21 reads or writes files in the storage area of ​​the storage in response to instructions from the host that uses the storage. The deduplication unit 22 performs deduplication processing on the files to be read or written so that the files are not duplicated in the storage area.

[0022] The duplication permission unit 23 also identifies the access status of the host to the file in the storage area. If the identified access status satisfies a set condition, the duplication permission unit 23 excludes the corresponding file from the target of the deduplication process.

[0023] In this way, the storage control device 20 performs deduplication on files stored in storage, just as in the past, but also performs deduplication under certain conditions. Therefore, the storage control device 20 can reduce unnecessary resource consumption in managing deduplication in storage.

[0024] Next, the configuration and functions of the storage control device 20 will be specifically described with reference to Fig. 2. Fig. 2 is a configuration diagram showing the configuration of an example of a storage control device in more detail.

[0025] 2, two storage control devices 20a and 20b, together with storage devices 31 to 34, constitute a file system 2. In the file system 2, for example, HDDs (Hard Disk Drives) and SSDs (Solid State Drives) are used as the storage devices 31 to 34. However, in the embodiment, the storage devices 31 to 34 are not limited to HDDs and SSDs as long as they are provided with readable and writable storage media.

[0026] In this way, general storage devices are used as the storage devices 31 to 34, but in this embodiment, to enable the above-mentioned deduplication, the storage device 31 is equipped with a deduplication hash table 312. The deduplication hash table 312 will be described later.

[0027] 2, hosts A to D use file system 2. Hosts A to D are virtual PCs constructed in a VDI environment. Of hosts A to D, hosts A and B access storage control device 20a, and hosts C and D access storage control device 20b.

[0028] 2, the storage control device 20a and the storage control device 20b have the same configuration. In the following description, when there is no need to distinguish between the storage control device 20a and the storage control device 20b, they are also referred to as "storage control device 20" as in FIG. 1.

[0029] 2, the storage control device 20 (20a and 20b) includes a host interface 24 and a disk interface 25 in addition to the file processing unit 21, deduplication unit 22, and duplication permission unit 23 described above. The host interface 24 is an interface for performing data communication with a host, which is a virtual PC. The disk interface 25 is an interface for performing data communication with the storage devices 31 to 34.

[0030] Here, the structure of data managed by the file system 2 will be described with reference to Figures 3 to 6. Figure 3 is a diagram showing an example of a directory managed by the file system. As shown in Figure 3, a directory area 311 is allocated in the storage area of ​​the storage device 31.

[0031] As shown in FIG. 3, in the directory area 311, file information entries 3111 and 3112 managed by the storage device exist for each file.

[0032] The file information entry 3111 stores a file identifier 31111, a logical address 31112, and a physical address 31113.

[0033] In a storage device with an enabled deduplication function, when deduplication is performed on a file-by-file basis, the hash value 31114 of the target file is stored in the file information entry 3111. In addition, the file information entry 3111 is allocated an area for storing control information 31115 of the file.

[0034] Additionally, in this embodiment, in order to enable the above-mentioned deduplication, the file information entry 3111 is also allocated a storage area for access information 31116 for analyzing information such as the access frequency of the file.

[0035] Similarly, the file information entry 3112 stores a file identifier 31121, a logical address 31122, a physical address 31123, a hash value 31124, file control information 31125, and access information 31126.

[0036] 3, only two file information entries are shown, but the embodiment is not limited to this. As long as there is room in the capacity of the directory area 311, as many file information entries as necessary are secured.

[0037] Fig. 4 is a diagram showing an example of the contents of the access information shown in Fig. 3. As shown in Fig. 4, the access information 31116 stores R / W (Read / Write) access frequency information 311161 of the target file, a new hash entry creation frequency after deduplication 311162, an existing hash entry update frequency after deduplication 311163, and other access frequency information 311164.

[0038] Also, the same information as the access information 31116 is stored in the access information 31126. Also, although not explicitly shown in Fig. 4, it is assumed that similar access information areas are allocated to other file information entries.

[0039] 5 is a diagram showing an example of a hash table for managing files to be deduplication-targeted when the deduplication function is enabled. As shown in Fig. 5, the deduplication hash table 312 stores a hash entry search database 3121 and hash entries 31221 to 31223.

[0040] Hash entry 31221 stores a hash value 312211, a number of duplicates 312212, a physical address 312213, and control information 312214. Hash entries 31222 and 31223 also store the same information as hash entry 31221. Although not explicitly shown in Fig. 5, it is assumed that any number of hash entries can be added as long as the capacity of the hash table allows.

[0041] Fig. 6 is a diagram showing an example of implementation of the hash entry search database shown in Fig. 5. A case where hash entries 31221, 31222, and 31223 are searched for using the hash entry search database 3121 shown in Fig. 6 will be described.

[0042] First, the deduplication unit 22 (see FIG. 2) performs a search from the hash table root 31211. Specifically, the deduplication unit 22 compares the hash value to be searched with each of the hash values ​​in the hash value ranges p, q, ... of the hash table root 31211.

[0043] Next, the deduplication unit 22 searches for the upper branch 312121 associated with the hit hash value range p.

[0044] Next, the deduplication unit 22 searches for lower branches linked to the hash value range pp, qq, ... of the upper branch 312121. Specifically, the deduplication unit 22 compares, for example, the hash value to be searched with each of the hash values ​​in the hash value range pp, qq, ... of the upper branch 312121, and if a hit is found in the hash value range pp, then it searches the lower branch 312131.

[0045] Furthermore, for example, if a hit is made in the hash value range ppp of the lower branch 312131, the deduplication unit 22 refers to the physical address of each hash entry recorded in the lower branch 312131 and compares the hash entries 31221, 31222, 31223, etc. with the hash value to be searched.

[0046] If the comparison reveals a hash entry that matches the hash value being searched for, the deduplication unit 22 executes deduplication processing based on the information in that hash entry. On the other hand, if there is no hash entry that matches the hash value being searched for, a new hash entry is added.

[0047] Although not shown in Figure 6, the hash entry search database 3121 also records upper branches from upper branch 312122 onwards (e.g., 312123, 312124, etc.) and lower branches from lower branch 312131 onwards (e.g., 312132, 312133, etc.).

[0048] Next, file updates by a host will be described with reference to Figs. 7 to 13. Fig. 7 is a diagram showing an example of a VDI environment. As shown in Fig. 7, hosts A to D used by users A to D, respectively, are clients (virtual PCs) A to D constructed in a VDI environment. Users A to D access their own VDI environments on storage system 2 (see Fig. 2) via their respective hosts. In the example of Fig. 7, user A accesses resources in VDI environment A within storage system 2 via host A111. The same is true for users B to D.

[0049] Fig. 8 is a diagram showing an example of the operation of a host when a file is copied locally in a VDI environment. In the example of Fig. 8, in each VDI environment, a development file A is copied from a development management server using a file management tool. Note that in the examples of Figs. 8 to 11, SVN is used as the file management tool, but the file management tool is not limited to SVN. In the embodiment, the use of other file management tools is not excluded.

[0050] In the example of Fig. 8, each of users A to D locally copies file A held by the development management server to their own VDI environment. In SVN, this operation is called "SVN checkout."

[0051] Figure 9 shows an example of the host's operation when a file is locally updated in a VDI environment. In the example of Figure 9, user D rewrites the code of file A, thereby updating file A to file A'. This rewrite operation is performed within the VDI environment of client D.

[0052] Figure 10 is a diagram showing the operations that follow the operations shown in Figure 9. After the operations shown in Figure 9 are performed, as shown in Figure 10, User D writes File A' to the development management server so that the revised File A' with the rewritten code is reflected on the development management server. In SVN, this operation is called an "SVN commit." The "SVN commit" operation updates File A on the development management server to File A'.

[0053] Fig. 11 is a diagram showing the operation following the operation shown in Fig. 10. When the operation shown in Fig. 10 is performed, users A to C each update file A to the latest state, creating file A', as shown in Fig. 11. This type of operation is called an "SVN update" in SVN.

[0054] Figure 12 shows an example of the host's behavior when a local file update is canceled in a VDI environment. As shown in Figure 12, user D cancels the code modification of file A and performs a process to revert file A' to the original file A. As a result, file A' on the development management server is also reverted to file A. In SVN, this type of operation is called "SVN change undo."

[0055] FIG. 13 is a diagram showing an example of host operation when local file updates are repeatedly performed in a VDI environment. In the example of FIG. 13, user D changes the code of file A multiple times. User D performs five updates: file A to file A', file A' to file A'', file A'' to file A''', file A''' to file A'''', file A'''' to file A'''', and file A'''' to file A'''''. In this case, the five updates to file A are closed within VDI environment D, so they do not affect the VDI environments of other users.

[0056] [Device operation] Next, the operation of the storage control device 20 will be described with reference to Figs. 14 to 17. Figs. 14 to 17 are flow diagrams showing an example of the operation of the storage control device. Figs. 14 to 17 show a series of consecutive operations. In the following description, Figs. 1 to 13 will be referenced as appropriate. In the embodiment, a storage control method is implemented by operating the storage control device 20. Therefore, the description of the storage control method will be replaced by the following description of the operation of the storage control device 20.

[0057] In the flow diagrams shown in FIGS. 14 to 17, when an alphabet and a number (such as C1) are written to the right of a box indicating a process, this indicates that the process indicated by the box is the subject of performance analysis.

[0058] First, FIG. 14 shows how the write process branches depending on whether deduplication is enabled or disabled when the storage control device 20 is instructed to write data from the host.

[0059] 14, first, the deduplication unit 22 reads the directory to check whether the data to be written is registered as an existing file (step S1). Next, the deduplication unit 22 determines whether the data to be written matches an existing identifier or logical address in the read directory (step S2).

[0060] If the result of the determination in step S2 is that there is no match, the deduplication unit 22 determines that it is a new write, and further determines whether deduplication is enabled or disabled (step S3). If the result of the determination in step S3 is that deduplication is disabled, a normal write process is executed at terminal F. On the other hand, if the result of the determination in step S3 is that deduplication is enabled, a new write process is executed at terminal B when deduplication is enabled.

[0061] If the result of the determination in step S2 is a match, that is, if an existing identifier in the directory is hit, the deduplication unit 22 determines that the process is to overwrite the existing file, and further determines whether deduplication is enabled or disabled (step S4). Then, if the result of the determination in step S4 is that deduplication is disabled, the normal write process of terminal F is executed.

[0062] On the other hand, if the result of the determination in step S4 is that deduplication is valid, the deduplication unit 22 calculates a hash value of the data to be written (step S5). Furthermore, the deduplication unit 22 compares the calculated hash value with a hash value in the directory (such as the hash value 311114 shown in FIG. 3) (step S6). Then, the deduplication unit 22 determines whether the calculated hash value matches the hash value in the directory (step S7) to check whether the data has been updated.

[0063] If the result of the determination in step S7 is that the calculated hash value matches the hash value in the directory (the hash value matches the value before the update), the deduplication unit 22 determines that there is a high possibility that the data has not been changed, and performs a comparison between the data to be written and the actual data (step S8).

[0064] Next, the deduplication unit 22 uses the comparison result of step S8 to determine whether the write data and the actual data match (step S9). If the determination result of step S9 shows that the write data and the actual data match, the write process is unnecessary, and the termination process of terminal G is executed. On the other hand, if the determination result of step S9 shows that the write data and the actual data do not match, the process branches to collision processing of terminal C. In a typical implementation method of deduplication storage, when the hash values ​​match but the actual data differ, this is called a "collision." The implementation of collision processing is similar to that of conventional deduplication storage, and is not specifically mentioned in this disclosure.

[0065] If the result of the determination in step S7 is that the calculated hash value does not match the hash value in the directory (the calculated hash value does not match the hash value stored in the directory), an update process for the data of terminal A is executed.

[0066] Fig. 15 shows the processing flow after branching to terminal A shown in Fig. 14. When the processing shown in Fig. 15 is executed, the calculated hash value does not match the hash value in the directory, so it can be determined that the data file to be written has been deduplicated.

[0067] Therefore, as shown in FIG. 15, first, the deduplication unit 22 obtains a hash value from the directory before the update (step A1).

[0068] Next, the deduplication unit 22 searches the hash table based on the hash value in the directory to identify the hash entry of the data before the update (step A2). Then, since the data has been updated, the deduplication unit 22 considers that one duplicate data item has been deleted, updates the hash entry, and decrements the number of duplicates of the hash entry (e.g., 312212 in FIG. 4) by one (step A3).

[0069] Next, the deduplication unit 22 determines whether the number of duplicates of the hash entry is 0 (step A4). If the result of the determination in step A4 is that the number of duplicates is 0, this means that there is no file information entry that uses the data to be written, so the deduplication unit 22 executes a hash entry deletion process (step A5) and further deletes the hash entry from the hash table (step A6).

[0070] If the result of the determination in step A4 is that the number of duplicates is not 0, new write processing is executed at terminal B when deduplication is enabled.

[0071] Fig. 16 shows the flow of the new data write process when deduplication is enabled. As shown in Fig. 16, when step A6 shown in Fig. 15 is executed, or when the determination in step A4 shows that the number of duplicates is not 0, the deduplication unit 22 searches the hash table using the hash entry search database 3121 based on the hash value of the write data from the host (step B1).

[0072] Next, the deduplication unit 22 determines whether there is a hash entry in the hash table that matches the hash value (step B2). Specifically, in step B2, the deduplication unit 22 references the hash entry and compares the data registered in the hash entry with the data to be written from the host.

[0073] If the result of the determination in step B2 shows that there is a hash entry with a matching hash value in the hash table, the deduplication unit 22 references the hash entry (step B3) and compares the entry with the actual data (step B4).Furthermore, based on the result of the comparison, the deduplication unit 22 determines whether there is matching actual data (step B5).

[0074] If the result of the determination in step B5 is that matching actual data exists, the deduplication unit 22 executes processing to update the hash entry (step B6). After this, processing is performed at terminal D. If the result of the determination in step B5 is that matching actual data does not exist, that is, if the hash values ​​match but the data comparison does not match, the deduplication unit 22 executes collision processing.

[0075] If the result of the determination in step B2 is that there is no hash entry in the hash table that matches the hash value, the deduplication unit 22 creates a new hash entry and performs processing to add the hash entry (step B7). Specifically, the deduplication unit 22 creates a new hash entry and adds it to the hash table. Thereafter, processing is performed at terminal E.

[0076] Figure 17 shows the processing flow for performance analysis of deduplication processing and post-processing such as writing actual data. In Figure 17, the processing within the dotted line box is particularly distinctive. The processing from terminal D is performed when a file that had been deduplication-depleted before is updated and the updated data hits an existing hash entry.

[0077] 17, first, at terminal D, the duplication permission unit 23 updates the "post-deduplication existing hash entry update frequency" (e.g., 311163 in FIG. 4) (step D1). Next, the duplication permission unit 23 performs a performance analysis calculation, which will be described later, to calculate the deduplication performance difference (step D2).

[0078] Next, the duplication permission unit 23 determines whether the calculated deduplication performance difference exceeds a threshold value (step D3). If the result of the determination in step D3 is that the deduplication performance difference exceeds the threshold value, the duplication permission unit 23 rewrites the control information (such as 31115 in FIG. 3) of the file information entry to disable deduplication of the file (step D4).

[0079] Since deduplication was disabled in step D4, the data to be written becomes newly written data, and therefore file processing unit 21 executes write processing w1 of the data to be written (step D5).

[0080] Next, if the result of the determination in step D3 is that the deduplication performance difference does not exceed the threshold value, and if step D5 has been executed, file processing unit 21 executes a directory update process (step D6).

[0081] In addition, the processing from terminal E is the processing that is performed when, when a file that had been de-duped before the update is updated, the updated data does not match the existing hash entry, new data is written, and a new hash entry is created at the same time.

[0082] 17, first, at terminal E, the duplication permission unit 23 updates the "post-duplication elimination new hash entry creation frequency" (e.g., 311162 in FIG. 3) (step E1). Next, the duplication permission unit 23 performs a performance analysis calculation, which will be described later, and calculates the deduplication performance difference (step E2).

[0083] Next, the duplication permission unit 23 determines whether the deduplication performance difference exceeds a threshold value (step E3). If the result of the determination in step E3 is that the deduplication performance difference exceeds the threshold value, the duplication permission unit 23 rewrites the control information (such as 31115 in FIG. 3) of the file information entry to disable deduplication of the file (step E4).

[0084] Since deduplication was disabled in step E4, the data to be written becomes newly written data, and therefore file processing unit 21 also executes steps D5 and D6 in this case.

[0085] In this way, in the processing of terminals E and F, since it is necessary to write new data, write processing w1 (step D5) of the actual data to be written is performed, and the process proceeds to directory update processing. On the other hand, in the processing of terminals D and G, since deduplication is performed, writing of actual data is not necessary, so write processing w1 of the actual data is not performed, and the process proceeds to directory update processing. Once the directory is updated, the write processing is completed.

[0086] Here, the content of processing for each operation mode when deduplication is performed will be described with reference to Figures 18 to 22. Figures 18 to 22 are diagrams showing an example of the content of processing for each operation mode when deduplication is performed.

[0087] FIG. 18 shows the process for new writing when there is no file information entry in the directory and the data to be written does not match any existing hash entry.

[0088] FIG. 19 shows the content of processing when a new write is performed when there is no file information entry in the directory and the data to be written hits an existing hash entry.

[0089] FIG. 20 shows the process when the file information entry on the directory indicates that there is data to be deduplied, and the data to be written matches the data before the update.

[0090] Figure 21 shows the processing that is performed when a file information entry in a directory indicates that there is data to be de-duped, the data to be written does not match the data before the update, and does not match any existing hash entries.

[0091] Figure 22 shows the processing that is performed when a file information entry in a directory indicates that there is data to be de-duped, the data to be written does not match the data before the update, and an existing hash entry is hit.

[0092] Next, the performance analysis process when deduplication is disabled will be described with reference to FIGS. Fig. 23 is a diagram showing a list of notations for the execution time of each process shown in Figs. 14 to 22. As shown in Fig. 23, the execution time of hash value calculation c1 is expressed as t(c1). Hereinafter, the execution time of process xx is expressed as t(xx). Furthermore, the value of t(xx) may be a fixed value at the time of implementation of the device, or may be corrected based on a measured value of the actual access time of the device.

[0093] Fig. 24 is a diagram showing a list of definitions of the execution time of each process shown in Fig. 14 to Fig. 22. As shown in Fig. 24, the time t(write-new) for normal writing without using deduplication is defined by the following equation 1.

[0094] (Number 1) t(write-new)=t(host-w)+t(file-sys)+t(disk-w)

[0095] In addition, in the above equation 1, "t(host-w)" indicates the time to receive write data from the host, and "t(file-sys)" indicates the time for common processing within the storage system 2. Furthermore, "t(host-w)" and "t(file-sys)" are estimated to be parameters specific to the storage system, independent of the file. For this reason, the performance index used for performance analysis is an index that removes "t(host-w)" and "t(file-sys)". Therefore, the above equation 1 can be rewritten as the following equation 2.

[0096] (Number 2) t'(write-new)=t(disk-w)

[0097] In the following, "t'(xxx)" in the above formula 2 will be expressed as the write load index in the storage system 2. As shown in FIG. 24, the write load index for each deduplication operation mode is as shown in the following formulas 3 to 7.

[0098] (Number 3) t'(dedup-same) = t(r5)+t(c2)

[0099] (Number 4) t'(dedup-new) = t(disk-w) + t(c1) +t(r1)+t(r2)+t(r3)+t(w5)+t(w3)

[0100] (Number 5) t'(dedup-hit) = t(c1)+t(r1)+t(r2)+t(r3)+t(r4)+t(w4)

[0101] (Number 6) t'(dedup-renew) = t(disk-w)+t(c1)+t(r1)x2+t(r2)x2+t(r3)x2+t(r4)+t(w4)x2+t(w5)

[0102] (Number 7) t'(dedup-other)= t(c1)+t(r1)x2+t(r2)x2+t(r3)x2+t(r4)x4+t(w4)x2

[0103] Figure 25 is a diagram showing the definition of parameters that represent the access frequency of each data write pattern when deduplication is enabled. As shown in Figure 25, the parameter that represents each access frequency indicates what percentage of all write processes corresponds to that operating mode. Therefore, the parameters that represent each access frequency are determined so that their total sum is 1 or less.

[0104] Fig. 26 is a diagram showing an example of a formula for calculating a performance index using the write load performance index shown in Fig. 24 and the access frequency parameter shown in Fig. 25. In Fig. 26, the performance index is calculated as a total performance index when deduplication is enabled by taking a weighted average of the write load performance index shown in Fig. 24 and the access frequency parameter shown in Fig. 25.

[0105] Fig. 27 is a diagram showing an example of a formula for calculating the deduplication performance difference. As shown in Fig. 27, the deduplication performance difference is calculated from the difference between the total performance index shown in Fig. 26 and a pre-calculated reference performance index. The reference performance index is calculated by taking a weighted average of a load performance index and an access frequency parameter for a pre-set normal write process.

[0106] Fig. 28 is a diagram showing another example of the formula for calculating the deduplication performance difference. In Fig. 28, the formula shown in Fig. 27 is simplified under certain conditions. The conditions are as follows: There is no need to overwrite data that has not changed. Some file management tools have a feature that prevents files that have not been changed from being overwritten.

[0107] Under the above conditions, the "dedup-same" operating mode, which overwrites data without making any changes, rarely exists, and "p(write-dedup-same)" can be considered to be 0 (zero). Furthermore, "dedup-new" and "dedup-hit" occur only once when a new file is created, and do not occur in subsequent file updates. Furthermore, for files that are updated frequently, the frequency of new data creation ("dedup-new") and "dedup-hit" can be considered relatively low, so these can also be considered to be 0 (zero). Therefore, based on the above assumptions, the calculation formula shown in Figure 28 holds. Furthermore, the deduplication performance difference "t'diff'" calculated using the simplified formula shown in Figure 28 will be specifically referred to as the deduplication effectiveness index.

[0108] The deduplication effectiveness index t'diff' is calculated as the difference between the weighted average of the execution time for each operating mode when deduplication is enabled and the normal write time when deduplication is not used. Therefore, if the value is negative or a small positive value, deduplication is deemed effective. Therefore, if the deduplication effectiveness index is a large positive value, the expected execution time is longer than normal write time, and it can be determined that enabling deduplication has caused a performance degradation.

[0109] Therefore, t'diff' is compared with a specific threshold, and if it is determined that t'diff' is greater than this specific threshold, a process is executed to cancel deduplication of the file (see FIG. 17). Note that although the specific threshold is clearly a positive value, the actual value is set appropriately depending on the implementation of the storage system.

[0110] In this manner, in the embodiment, files whose deduplication performance difference exceeds the threshold, i.e., files that are frequently updated, are excluded from deduplication. Therefore, for example, files that are rarely modified or altered remain deduplication targets. Therefore, in the embodiment, the occurrence of situations in which resources for managing the deduplication function are wasted is suppressed.

[0111] [program] In the embodiments, the program may be a program that causes a computer to execute the steps shown in FIGS. 14 to 17. By installing and executing this program on a computer, the storage control device 20 and the storage control method can be realized. In this case, the processor of the computer functions and performs processing as a file processing unit 21, a deduplication unit 22, and a duplication permission unit 23. Examples of the computer include a general-purpose PC, a server device, a smartphone, and a tablet terminal device. Furthermore, the computer may be a virtual PC built in a VDI environment.

[0112] In the embodiment, the program may be executed by a computer system constructed by multiple computers. In this case, for example, each computer may function as one of the file processing unit 21, the deduplication unit 22, and the duplication permission unit 23.

[0113] [Physical configuration] Here, a computer that implements the storage control device 20 by executing the program in the embodiment will be described with reference to Fig. 29. Fig. 29 is a block diagram showing an example of a computer that implements the storage control device.

[0114] 29, a computer 210 includes a CPU (Central Processing Unit) 211, a main memory 212, a storage device 213, an input interface 214, a display controller 215, a data reader / writer 216, and a communication interface 217. These components are connected to each other via a bus 121 so as to be able to communicate data with each other.

[0115] Furthermore, the computer 210 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to or instead of the CPU 211. In this aspect, the GPU or FPGA can execute the programs in the embodiments.

[0116] The CPU 211 loads a program in the embodiment, which is composed of a group of codes and stored in the storage device 213, into the main memory 212 and executes each code in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).

[0117] Moreover, the program in the embodiment is provided in a state stored in a computer-readable recording medium 220. The program in the embodiment may be distributed over the Internet connected via the communication interface 117.

[0118] Specific examples of the storage device 213 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 214 mediates data transmission between the CPU 211 and input devices 218 such as a keyboard and a mouse. The display controller 215 is connected to a display device 219 and controls the display on the display device 219.

[0119] The data reader / writer 216 mediates data transmission between the CPU 211 and the recording medium 220, reads programs from the recording medium 220, and writes processing results from the computer 210 to the recording medium 220. The communication interface 217 mediates data transmission between the CPU 211 and other computers.

[0120] Specific examples of the recording medium 220 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as flexible disks, or optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0121] The storage control device 20 can be realized not by a computer with a program installed, but by hardware corresponding to each unit, such as an electronic circuit. Furthermore, the storage control device 20 may be partially realized by a program and the remaining part by hardware. In the embodiments, the computer is not limited to the computer shown in FIG. 29.

[0122] Some or all of the above-described embodiments can be expressed by (Supplementary Note 1) to (Supplementary Note 12) described below, but are not limited to the following descriptions.

[0123] (Appendix 1) a file processing unit that reads or writes files in a storage area of ​​the storage in response to an instruction from a host that uses the storage; a deduplication unit that performs deduplication processing on the file to be read or written so that the file does not exist duplicated in the storage area; a duplication permission unit that identifies an access status of the file in the storage area, and, if the identified access status satisfies a set condition, excludes the corresponding file from the target of the deduplication process; A storage control device comprising:

[0124] (Appendix 2) the duplication permission unit calculates a deduplication performance difference based on the access frequency of the file to identify the access status, and if the deduplication performance difference exceeds a threshold, excludes the corresponding file from the deduplication processing. 2. The storage control device according to claim 1.

[0125] (Appendix 3) the duplication permission unit determines the access frequency of the file from access information about the file recorded in the storage; 3. The storage control device according to claim 2.

[0126] (Appendix 4) The host is a virtual computer built on a virtual desktop infrastructure, and the file is used by multiple hosts. 2. The storage control device according to claim 1.

[0127] (Appendix 5) a file processing step of reading or writing a file in a storage area of ​​the storage in response to an instruction from a host that uses the storage; a deduplication step of performing a deduplication process on the file to be read or written so that the file does not exist duplicated in the storage area; a duplication allowing step of identifying an access status of the file in the storage area, and excluding the corresponding file from the target of the deduplication process if the identified access status satisfies a set condition; A storage control method comprising:

[0128] (Appendix 6) In the duplication permission step, the access status is identified by calculating a deduplication performance difference based on an access frequency of the file, and if the deduplication performance difference exceeds a threshold, the corresponding file is excluded from the deduplication processing. 6. The storage control method according to claim 5.

[0129] (Appendix 7) In the overlap permission step, an access frequency of the file is calculated from access information about the file recorded in the storage. 7. The storage control method according to claim 6.

[0130] (Appendix 8) The host is a virtual computer built on a virtual desktop infrastructure, and the file is used by multiple hosts. 6. The storage control method according to claim 5.

[0131] (Appendix 9) On the computer, a file processing step of reading or writing a file in a storage area of ​​the storage in response to an instruction from a host that uses the storage; a deduplication step of performing a deduplication process on the file to be read or written so that the file does not exist duplicated in the storage area; a duplication allowing step of identifying an access status of the file in the storage area, and excluding the corresponding file from the target of the deduplication process if the identified access status satisfies a set condition; A program that executes.

[0132] (Appendix 10) In the duplication permission step, the access status is identified by calculating a deduplication performance difference based on an access frequency of the file, and if the deduplication performance difference exceeds a threshold, the corresponding file is excluded from the deduplication processing. 10. The program described in Appendix 9.

[0133] (Appendix 11) In the overlap permission step, an access frequency of the file is calculated from access information about the file recorded in the storage. 10. The program described in Appendix 10.

[0134] (Appendix 12) The host is a virtual computer built on a virtual desktop infrastructure, and the file is used by multiple hosts. 10. The program described in Appendix 9. [Industrial Applicability]

[0135] As described above, according to the present disclosure, it is possible to prevent unnecessary consumption of resources in managing deduplication in storage. The present disclosure is useful for computer systems in which storage is shared by multiple hosts. [Explanation of symbols]

[0136] 2 File Systems 20, 20a, 20b Storage control device 21 File Processing Unit 22 Deduplication section 23 Duplicate Permission Section 24 host interfaces 25 Disk Interface 31~34 Storage device 210 Computer 211 CPU 212 main memory 213 Storage device 214 input interface 215 Display Controller 216 Data Reader / Writer 217 Communication Interface 218 Input Devices 219 Display Device 220 Recording Media 221 Bus

Claims

1. a file processing unit that reads or writes files in a storage area of ​​the storage in response to an instruction from a host that uses the storage; a deduplication unit that performs deduplication processing on the file to be read or written so that the file does not exist duplicated in the storage area; a duplication permission unit that identifies an access status of the file in the storage area, and, if the identified access status satisfies a set condition, excludes the corresponding file from the target of the duplication elimination process; Equipped with The overlap permission unit A performance index is calculated by taking a weighted average of the write load performance index for each operation mode of deduplication and an access frequency parameter that indicates the access frequency of the data write pattern when deduplication is enabled, and further, A reference performance index is calculated by taking a weighted average of a load performance index and an access frequency parameter in a write process during normal operation, which is set in advance, and calculating a difference between the performance index and the reference performance index as a deduplication performance difference to identify the access status; A storage control device characterized in that, when the deduplication performance difference exceeds a threshold, the corresponding file is excluded from the deduplication processing.

2. the duplication permission unit determines the access frequency of the file from access information about the file recorded in the storage; The storage control device according to claim 1 .

3. The host is a virtual computer built on a virtual desktop infrastructure, and the file is used by multiple hosts. The storage control device according to claim 1 .

4. a file processing step of reading or writing a file in a storage area of ​​the storage in response to an instruction from a host that uses the storage; a deduplication step of performing a deduplication process on the file to be read or written so that the file does not exist duplicated in the storage area; a duplication allowing step of identifying an access status of the file in the storage area, and excluding the corresponding file from the target of the deduplication process if the identified access status satisfies a set condition; and In the overlap permission step, A performance index is calculated by taking a weighted average of the write load performance index for each operation mode of deduplication and an access frequency parameter that indicates the access frequency of the data write pattern when deduplication is enabled, and further, A reference performance index is calculated by taking a weighted average of a load performance index and an access frequency parameter in a write process during normal operation, which is set in advance, and calculating a difference between the performance index and the reference performance index as a deduplication performance difference to identify the access status; A storage control method comprising: excluding the corresponding file from the deduplication processing when the deduplication performance difference exceeds a threshold value.

5. On the computer, a file processing step of reading or writing a file in a storage area of ​​the storage in response to an instruction from a host that uses the storage; a deduplication step of performing a deduplication process on the file to be read or written so that the file does not exist duplicated in the storage area; a duplication allowing step of identifying an access status of the file in the storage area, and excluding the corresponding file from the target of the deduplication process if the identified access status satisfies a set condition; Execute In the overlap permission step, A performance index is calculated by taking a weighted average of the write load performance index for each operation mode of deduplication and an access frequency parameter that indicates the access frequency of the data write pattern when deduplication is enabled, and further, A reference performance index is calculated by taking a weighted average of a load performance index and an access frequency parameter in a write process during normal operation, which is set in advance, and calculating a difference between the performance index and the reference performance index as a deduplication performance difference to identify the access status; If the deduplication performance difference exceeds a threshold, the program excludes the corresponding file from being a target of the deduplication process.

Images