Storage device and data processing method
By combining processors and accelerators in the storage device, it is divided into small processing units for repeated exclusion and large processing units for compression, and solves the problems of I/O performance reduction and poor data reduction effects in the prior art, and achieves efficient data Cut and low load handling.
Patent Information
- Application Number
- CN202110188671.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-09-17
- Filing Date
- 2021-02-19
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-02-19
AI Technical Summary
In the prior art, in the storage device that processes a large amount of data, the combination of repeated exclusion and compression will lead to a decrease in I/O performance, and the data reduction effect varies depending on the processing unit size, making it difficult to take into account both efficient reduction and low-load processing.
Design a storage device, including a processor, accelerator and memory, divide the data into smaller processing units for repeated exclusion, and compress the remaining data in larger processing units, and use the accelerator to perform high-speed compression and decompression processing. Adjust processing units to improve data reduction effect.
It realizes that without reducing I/O performance, data reduction effect, processor load, and overall performance of storage devices are improved.
Smart Images

Figure CN114201105B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a storage device with a capacity reduction function. Background Art
[0002] In recent years, technologies that generate new value by analyzing large amounts of data, such as IoT (Internet of Things) and AI (Artificial Intelligence), have become popular. Storage devices that store large amounts of data require low data storage costs (price per unit capacity) and high I / O (Input / Output) performance.
[0003] The data storage cost is calculated as the product of the bit cost (price per unit capacity) and the amount of data stored. As technologies for reducing the data storage cost by reducing the amount of data stored, there are deduplication and compression. Deduplication is a function of searching for the same data in the storage device and deleting the data of one party and referring to the data of the other party to control. Compression is a function of replacing the bit string of data with a shorter bit string by encoding it.
[0004] In recent years, storage devices that combine deduplication and compression to keep data at a low cost have become popular.
[0005] However, encoding of a bit string is a process with a large computational load, and thus may cause a decrease in I / O performance in a storage device that processes a large amount of data.
[0006] As a countermeasure for the reduction of I / O performance, for example, there is known a technology described in Patent Document 1. Patent Document 1 discloses a system in which an accelerator capable of compressing and decompressing data at high speed using dedicated hardware is mounted on a CPU bus, and compression and decompression of data stored in a memory and data input and output on the bus are offloaded to the accelerator, thereby reducing the load on the CPU.
[0007] Prior art literature
[0008] Patent Literature
[0009] Patent Document 1: U.S. Patent Publication No. 2017 / 0147624 Summary of the invention
[0010] Problems to be solved by the invention
[0011] The data reduction effects of deduplication and compression vary depending on the size of the processing unit.
[0012] In deduplication, the smaller the processing unit is, the higher the data reduction effect is. This is because when the processing unit is small, the probability of the storage device storing the same data is high. For example, when there are data such as ABCD and DCBA, if the processing unit is 4 characters, the two data are regarded as different data and deduplication is not performed. On the other hand, if the processing unit is 1 character, each part of A, B, C, and D is regarded as the same data and deduplication can be performed.
[0013] On the other hand, in compression, the larger the processing unit, the higher the data reduction effect. For example, assuming there are eight 8kB data, the data reduction rate is higher when the 64kB data formed by combining the eight data is compressed than when each data is compressed individually. This is because the reversible compression algorithm used in the storage device is generally a sliding dictionary method, and the larger the compression unit, the wider the dictionary search space, and the higher the probability of string consistency.
[0014] In order to achieve a higher data reduction effect, it is necessary to combine deduplication with a small processing unit and compression with a large processing unit. First, in the data string, the same data is searched in a smaller processing unit to perform deduplication. Then, the data string except the data deleted by deduplication is compressed. Here, in order to exclude the data deleted by deduplication, the processor needs to perform operations on the data string, which has the problem of reducing I / O performance.
[0015] Technical solutions to solve problems
[0016] A representative example of the invention disclosed in the present application is shown as follows. A storage device having a processor, an accelerator, a memory, and a storage component, wherein the processor determines compression target data and sends a compression instruction including information about the compression target data to the accelerator, wherein the compression target data is data to be stored in the storage component among the data stored in the memory, and the accelerator reads a plurality of continuous data from the memory, and compresses a plurality of compression target data except for non-compression target data among the plurality of data based on the information about the compression target data received from the processor, to generate compressed data to be stored in the storage component.
[0017] Effects of the Invention
[0018] According to the present invention, a storage device with a good data reduction effect can be realized without reducing I / O performance. Other problems, structures and effects than those described above will be explained by describing the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 This is a diagram for explaining an overview of processing executed by the storage device of the first embodiment.
[0020] Figure 2 This is a diagram for explaining an overview of processing executed by the storage device of the first embodiment.
[0021] Figure 3 This is a diagram showing an example of the structure of the storage device of the first embodiment.
[0022] Figure 4 This is a diagram showing an example of compressed data management information according to the first embodiment.
[0023] Figure 5 This is a diagram showing an example of deduplication data management information according to the first embodiment.
[0024] Figure 6 This is a flowchart for explaining an example of data reduction processing executed by the processor of the first embodiment.
[0025] Figure 7 This is a flowchart for explaining an example of compression processing performed by the accelerator of the first embodiment.
[0026] Figure 8 This is a flowchart for explaining an example of a data string read process executed by the processor of the first embodiment.
[0027] Fig. 9 This is a flowchart illustrating an example of decompression processing performed by the accelerator of the first embodiment.
[0028] Fig.10 This is a diagram for explaining an overview of processing executed by the storage device of the second embodiment.
[0029] Fig.11 This is a diagram for explaining an overview of processing executed by the storage device of the second embodiment.
[0030] Fig.12 This is a diagram showing an example of compressed data management information according to the second embodiment.
[0031] Fig.13 This is a diagram showing an example of host data management information in the second embodiment.
[0032] Fig.14A This is a flowchart for explaining an example of data reduction processing executed by the processor of the second embodiment.
[0033] Fig. 14B This is a flowchart for explaining an example of data reduction processing executed by the processor of the second embodiment.
[0034] Fig.15 This is a flowchart illustrating an example of compression processing performed by the accelerator of the second embodiment.
[0035] Fig.16This is a flowchart illustrating an example of garbage collection performed by the processor of the second embodiment. DETAILED DESCRIPTION
[0036] The following are embodiments of the present invention using the accompanying drawings. However, the present invention is not limited to the contents of the embodiments shown below. The specific structure can be changed within the scope of the idea or purpose of the present invention, which is easy to understand for practitioners in this industry.
[0037] In the configuration of the invention described below, the same or similar configurations or functions are denoted by the same reference numerals, and duplicate descriptions are omitted.
[0038] The terms “first”, “second”, “third”, etc. in this specification and the like are used to identify components, and do not necessarily limit the number or order.
[0039] [Example 1]
[0040] First, use Figure 1 and Figure 2 An overview of processing executed by the storage device 100 according to the first embodiment will be described. Figure 1 and Figure 2 This is a diagram for explaining an overview of processing executed by the storage device 100 according to the first embodiment.
[0041] The storage device 100 is a host 200 (refer to Figure 3 ) A device for providing a storage area, comprising a storage controller 101 including a processor 111, an accelerator 112 and a memory 113; and a storage medium 102. The host 200 recognizes the provided storage area as a logical volume, and uses the volume to read and write data.
[0042] First, use Figure 1 The outline of the write process executed by the storage device 100 of the first embodiment will be described. Here, the process in which the host 200 writes a data string consisting of a plurality of data arranged in address (logical address) order to a volume will be described.
[0043] First, the data string written by the host 200 is stored in the memory 113 . Figure 1 In the example, it is assumed that after writing the data string XBYZ, the data string ABCD is written.
[0044] The data string XBYZ is a data string consisting of data X, B, Y, and Z, and the data string ABCD is a data string consisting of data A, B, C, and D. The data in the data string are arranged in the order of logical addresses.
[0045] After storing the data string ABCD in the memory 113, the processor 111 compares the data string XBYZ stored in the memory 113 with the data string ABCD according to a predetermined processing unit. The processing unit is, for example, the minimum data size of the data string, that is, the data is compared in units such as data X and A. Because the result of the above comparison is that the data B of the data string ABCD is consistent with the data B of the other data string XBYZ, the processor 111 determines that the data B is data that does not need to be stored in the storage medium 102. The processor 111 generates information indicating that the data A, C, and D are compression target data.
[0046] The processor 111 instructs the accelerator 112 to compress the address of the data string ABCD in the memory 113 and the information on the compression target data. Here, the information on the compression target data is, for example, a list of a bit map and an address range. The bit map is a bit string with the same number of bits as the number of data constituting the data string, and 1 is stored in the bit corresponding to the compression target data.
[0047] The accelerator 112 obtains the data string ABCD from the memory 113 based on the address of the data string ABCD. In addition, the accelerator 112 obtains data A, C, and D from the data string ABCD based on the information about the compression target data, arranges the obtained data A, C, and D in address order to generate a data string ACD, and compresses the data string ACD based on the compression algorithm to generate compressed data acd. The accelerator 112 stores the compressed data acd in the memory 113. Afterwards, the processor 111 stores the compressed data acd in the memory 113 in the storage medium 102.
[0048] In deduplication, the processor 111 searches for duplicate data in smaller processing units. Thus, duplicate data can be efficiently searched and deleted, so that the data reduction effect can be improved. In compression, the accelerator 112 compresses the data string generated by arranging the compression target data, that is, the data (data string) of a larger processing unit than deduplication. Thus, compressed data with a high compression rate can be generated, so that the data reduction effect can be improved.
[0049] The above is an overview of the write process.
[0050] Next, use Figure 2 The following describes an overview of the read process executed by the storage device 100 of Embodiment 1. Here, the process is described when the host 200 sends a read request for the data string ABCD to the storage device 100. The read request includes the logical address of the data string ABCD in the volume.
[0051] The processor 111 reads the data B deleted by the deduplication processing from the storage medium 102 and stores it in the memory 113. In addition, the processor 111 reads the compressed data acd from the storage medium 102 and stores it in the memory 113. The processor 111 indicates the address of the compressed data acd in the memory 113, the storage location (address) of the data string ABCD in the memory 113, and the information about the configuration of the decompressed data A, C, and D to the accelerator 112, and decompresses. Here, the information about the configuration of the decompressed data is, for example, a list of a bit map and an address. The bit map is a bit string with the same number of bits as the number of data constituting the data string, and 1 is stored in the bit corresponding to the position of the configuration data.
[0052] The accelerator 112 obtains the compressed data acd from the memory 113 based on the address of the compressed data acd, decompresses the compressed data acd and generates a data string ACD. In addition, the accelerator 112 stores the data A, C, and D of the data string ACD at a predetermined address of the memory 113 based on the information about the arrangement of the decompressed data.
[0053] For example, when the information on the configuration of the decompressed data is a bitmap, the accelerator 112 stores the data in the data string ACD at the position corresponding to the bit storing 1. That is, the accelerator 112 stores data A at the first position of the configuration location of the memory 113 of the data string ABCD, stores data C at the third position, and stores data D at the fourth position.
[0054] In addition, after the data B is stored in the memory 113, the data A, C, and D are stored in the memory 113, but the order of storing the data in the memory 113 may be different.
[0055] The above is an overview of the reading process.
[0056] Next, the details of the storage device 100 according to the first embodiment will be described.
[0057] Figure 3 This is a diagram showing an example of the structure of the storage device 100 according to the first embodiment.
[0058] The storage device 100 is connected to a host 200 providing a storage area via a network such as a SAN (Storage Area Network) or a LAN (Local Area Network). The host 200 is, for example, a personal computer or a smartphone.
[0059] The storage device 100 includes a storage controller 101 and a storage medium 102 .
[0060] The storage controller 101 executes processing corresponding to the command. The storage controller 101 includes a processor 111, an accelerator 112, a memory 113, and a host interface 114. Each hardware is connected to each other via an internal network.
[0061] The host interface 114 is an interface for connecting to the host 200. The host interface 114 performs transmission and reception processing of commands and data between the storage device 100 and the host 200.
[0062] The processor 111 executes the program stored in the memory 113. The processor 111 of the first embodiment functions as a calculation unit or a control unit that executes various processes in accordance with commands.
[0063] The accelerator 112 is a characteristic element of the present invention, and executes compression processing and decompression processing according to the instruction of the processor 111. The accelerator 112 is hardware designed to execute compression processing and decompression processing at high speed, and is implemented using, for example, FPGA (Field Programmable Gate Array).
[0064] The memory 113 is a storage device such as DRAM (Dynamic Random Access Memory), and stores programs executed by the processor 111 and information used by the programs. In addition, the memory 113 is also used as a work area used by the programs. The memory 113 may be either a volatile memory or a nonvolatile memory.
[0065] The memory 113 of this embodiment stores the compressed data management information 300 and the duplicate elimination data management information 310. In addition, the memory 113 stores a program (omitted from the figure) that implements the operation unit or the control unit. For details of the compressed data management information 300 and the duplicate elimination data management information 310, use Figure 4 and Figure 5 Provide explanation.
[0066] The storage medium 102 is a non-volatile storage device. The storage medium 102 is, for example, a semiconductor memory such as a HDD (Hard Disk Drive) or an SSD (Solid State Drive). The storage medium 102 is connected to the storage controller 101 via an internal network, and performs the sending and receiving processing of commands between the processor 111 and the sending and receiving processing of data between the memory 113. In addition, the storage medium 102 is directly connected to the storage controller 101 via the internal network, but can also be connected to the storage controller 101 via an interface for the storage medium connected to the internal network.
[0067] Figure 4 This is a diagram showing an example of compressed data management information 300 according to the first embodiment.
[0068] The compressed data management information 300 is information for managing compressed data. The compressed data management information 300 stores entries composed of a logical address 301, a physical address 302, and a data size 303. One entry corresponds to one compressed data.
[0069] The logical address 301 is a field for storing a logical address indicating a storage location of a data string in a volume provided to the host 200 .
[0070] The physical address 302 is a field for storing an address indicating a storage location of compressed data in the storage device 100. Here, the compressed data is data generated by compressing a data string after performing deduplication processing.
[0071] The data size 303 is a field for storing the data size of the compressed data.
[0072] The data size of the compressed data depends on the size of the original data string and the stored value, etc. Therefore, the size of each compressed data is generally different. In order to improve the data reduction effect, the compressed data needs to be stored in the storage medium 102 without gaps. Therefore, the processor 111 uses the compressed data management information 300 to control the configuration of the compressed data in the storage medium 102.
[0073] In addition, instead of the physical address of the storage medium 102, it can be the address provided by the storage medium 102 to the storage controller 101, or the address of a RAID (Redundant Arrays of Inexpensive Disks) volume generated using multiple storage media 102, or the address of an external storage device 100 provided by a capacity virtualization mechanism.
[0074] Figure 5 This is a diagram showing an example of the deduplication data management information 310 according to the first embodiment.
[0075] The deduplication data management information 310 is information for managing data deleted from the data string due to deduplication. The deduplication data management information 310 stores entries composed of a logical address 311 and a reference logical address 312. One entry corresponds to one piece of data deleted due to deduplication.
[0076] The logical address 311 is a field for storing an address indicating a storage location of data deleted from a data string due to deduplication in a volume.
[0077] The reference logical address 312 is a field for storing an address indicating a storage location of data in the volume that matches the deleted data.
[0078] The processing unit of duplicate elimination is smaller than the processing unit of compression, so the data included in a certain data string may be consistent with the data included in other data strings.
[0079] Figure 6 This is a flowchart for explaining an example of data reduction processing executed by the processor 111 of the first embodiment.
[0080] The processor 111 performs data reduction processing when the write processing is executed or periodically. The write processing is executed when the storage device 100 receives a data string from the host 200. Here, it is assumed that the data reduction processing is executed when the write processing is executed. In addition, during the write processing, the storage device 100 stores the data string in the memory 113 and sends a completion notification of the write processing to the host 200.
[0081] Processor 111 divides the data string based on the processing unit of duplicate elimination (step S101). At this time, processor 111 initializes information on compression target data. For example, processor 111 generates the same number of bit strings (bitmaps) as the number of divided data.
[0082] Next, the processor 111 selects target data from the divided data (step S102).
[0083] Next, the processor 111 searches for data that matches the target data (step S103).
[0084] For example, the processor 111 calculates a search keyword of a certain size called a digital fingerprint based on the target data and compares it with the digital fingerprints of other data. If there is data with the same digital fingerprint as the target data, the processor 111 decompresses the retrieved data and compares the target data with the decompressed data.
[0085] In addition, the data to be compared are the data stored in the memory 113 and the storage medium 102 .
[0086] Next, the processor 111 determines whether or not there is data that matches the target data based on the search result (step S104 ).
[0087] If there is data that matches the target data, the processor 111 registers an entry about the target data in the deduplication data management information 310 (step S105 ). Thereafter, the processor 111 proceeds to step S107 .
[0088] Specifically, the processor 111 adds an entry to the deduplication data management information 310 , stores the logical address of the target data in the logical address 311 of the added entry, and stores the logical address of data that matches the target data in the reference logical address 312 of the added entry.
[0089] In addition, the logical address of the target data can be calculated based on the address of the data string including the target data, the order (offset) from the beginning of the target data, and the processing unit of duplication elimination.
[0090] If there is no data that matches the target data, the processor 111 registers the target data as compression target data (step S106 ). Thereafter, the processor proceeds to step S107 .
[0091] For example, the processor 111 sets 1 in the bit corresponding to the target data in the bitmap.
[0092] In step S107, the processor 111 determines whether the processing of all the data in the data string has been completed (step S107).
[0093] If the processing of all the data in the data string has not been completed, the processor 111 returns to step S102 and performs the same processing.
[0094] When the processing of all the data in the data string is completed, the processor 111 sends a compression instruction including the storage location (address) of the data string in the memory 113 and information about the compression target data to the accelerator 112 (step S108). The processor 111 shifts to a waiting state until receiving a completion notification from the accelerator 112. In addition, the completion notification includes the storage location (address) of the compressed data in the memory 113 and the data size of the compressed data.
[0095] When receiving the completion notification from the accelerator 112, the processor 111 determines the storage location of the compressed data in the storage medium 102 (step S109), and then ends the data reduction process. Specifically, the following process is performed.
[0096] (S109-1) The processor 111 refers to the compressed data management information 300 and searches for an entry storing the logical address of the data string in the logical address 301. If there is no corresponding entry, the processor 111 adds an entry to the compressed data management information 300 and stores the logical address of the data string in the logical address 301 of the added entry.
[0097] (S109-2) The processor 111 refers to the physical address 302 of the compressed data management information 300 to identify the end position (address) of the storage area storing the compressed data of the storage medium 102. The processor 111 determines the address of the storage area storing the compressed data based on the address.
[0098] (S109-3) The processor 111 stores the address determined in the physical address 302 of the entry storing the logical address of the data string in the logical address 301, and stores the notified data size of the compressed data in the data size 303 of the entry. Thus, the compressed data is stored compactly in the storage area of the storage medium 102.
[0099] The above is the description of the process of step S109.
[0100] When the data reduction process is performed periodically, the processor 111 performs Figure 6 Processing described in .
[0101] After executing the data reduction process, the processor 111 stores the compressed data at a determined address of the storage medium 102 at an arbitrary timing. For example, the data reduction process and the compressed data storage process may be continuously performed, or the compressed data storage process may be periodically performed. When the compressed data storage process is periodically performed, the processor 111 retrieves the compressed data stored in the memory 113 and stores the retrieved compressed data in the storage medium 102.
[0102] Figure 7 This is a flowchart for explaining an example of compression processing performed by the accelerator 112 of the first embodiment.
[0103] When receiving a compression instruction from the processor 111 , the accelerator 112 starts the compression process.
[0104] The accelerator 112 acquires the data string from the storage location (address) of the data string in the memory 113 notified by the processor 111 (step S201 ).
[0105] The accelerator 112 extracts the compression target data from the data string based on the information on the compression target data notified by the processor 111 (step S202 ).
[0106] For example, when the information on the compression target data is a bitmap, the accelerator 112 extracts data at positions corresponding to bits set to 1 as the compression target data.
[0107] The accelerator 112 generates a data string by arranging the compression target data in the order of logical addresses (step S203).
[0108] Next, the accelerator 112 applies a compression algorithm to the generated data string, thereby generating compressed data (step S204 ).
[0109] Next, the accelerator 112 stores the compressed data in the memory 113 (step S205 ).
[0110] Next, the accelerator 112 sends a completion notification including the storage location (address) of the compressed data in the memory 113 and the data size of the compressed data to the processor 111 (step S206 ). Thereafter, the accelerator 112 ends the compression process.
[0111] The accelerator 112 can reduce the processing load of the processor 111 by compressing the data string.
[0112] Figure 8 This is a flowchart for explaining an example of a data string read process executed by the processor 111 of the first embodiment.
[0113] When the storage device 100 receives a data read request from the host 200 or reads data predicted based on the tendency of the data read request in advance, the storage device 100 starts a data string read process.
[0114] Here, the reading process is described by taking the case where a data reading request is received as an example. In addition, the data reading request includes the logical address of the data string in the volume.
[0115] The processor 111 generates an address list of data constituting the data string based on the logical address, the size of the data string processed by the host 200, and the processing unit for duplicate elimination (step S301). The address list is a list that stores the starting address (logical address) of the data constituting the data string. At this time, the processor 111 determines the configuration location (address range) of the data string in the memory 113. In addition, the processor 111 initializes information on the configuration of the data constituting the decompressed data string. For example, the processor 111 generates the same number of bit strings (bitmaps) as the number of addresses registered in the address list.
[0116] Next, the processor 111 selects a destination address from the list (step S302).
[0117] The processor 111 refers to the deduplication data management information 310 and determines whether the data corresponding to the destination address has been deleted due to deduplication (step S303).
[0118] Specifically, processor 111 refers to logical address 311 and searches for entries that set logical addresses included in the address range determined by the destination address and the size of data (processing unit for deduplication). If there is an entry that meets the above conditions, processor 111 determines that the target data is deleted due to deduplication.
[0119] When it is determined that the data corresponding to the destination address is data to be deleted due to deduplication, the processor 111 reads the data corresponding to the destination address and stores it in the memory 113 (step S304). Thereafter, the processor 111 proceeds to step S306. Specifically, the following processing is performed.
[0120] ( S304 - 1 ) The processor 111 determines the location (address) of data corresponding to the destination address in the memory 113 based on the size of the data string processed by the host 200 , the logical address of the data string, and the logical address of the target data.
[0121] ( S304 - 2 ) The processor 111 determines whether the data corresponding to the reference logical address 312 of the entry retrieved in step S303 is stored in the memory 113 .
[0122] (S304-3) When the data is stored in the memory 113, the processor 111 copies the data to the determined arrangement location.
[0123] (S304-4) When the data is not stored in the memory 113, the processor 111 refers to the compressed data management information 300 and retrieves the entry in the logical address 301 that stores the start address of the address range including the retrieved address. The processor 111 reads the compressed data from the storage medium 102 based on the address stored in the physical address 302 of the retrieved entry, and stores it in the memory 113. The processor 111 sends a decompression instruction including the storage location (address) of the compressed data in the memory 113 and the configuration location of the decompressed data string to the accelerator 112. When receiving a completion notification from the accelerator 112, the processor 111 copies the data corresponding to the destination address included in the decompressed data string to the configuration location.
[0124] Furthermore, when receiving a decompression instruction, the accelerator 112 decompresses the compressed data and stores the decompressed data string in a designated location in the memory 113 .
[0125] The above is the description of the process of step S304.
[0126] When it is determined that the data corresponding to the destination address is not data to be deleted due to deduplication, the processor 111 registers the destination address as the data arrangement position (step S305 ). Thereafter, the processor 111 proceeds to step S306 .
[0127] Specifically, the processor 111 sets 1 in the bit corresponding to the destination address in the bitmap.
[0128] In step S306, the processor 111 determines whether the processing of all addresses in the address list has been completed (step S306).
[0129] If it is determined that the processing of all the addresses in the address list has not been completed, the processor 111 returns to step S302 and executes the same processing.
[0130] When it is determined that the processing of all addresses in the address list has been completed, the processor 111 reads the compressed data related to the data string from the storage medium 102 (step S307).
[0131] Specifically, processor 111 refers to compressed data management information 300 and searches for an entry storing an address in the address range specified by the read request in logical address 301. Processor 111 reads compressed data from storage medium 102 based on the address stored in physical address 302 of the searched entry and stores it in memory 113.
[0132] Next, the processor 111 sends a decompression instruction including the storage location (address) of the compressed data in the memory 113, the configuration location (address) of the data string, and information indicating the configuration location of each data of the data string to the accelerator 112 (step S308). Thereafter, the processor 111 performs a read process of the data string.
[0133] After receiving the completion notification from the accelerator 112 , the processor 111 sends the data string to the host 200 at an arbitrary timing.
[0134] Fig. 9 This is a flowchart illustrating an example of decompression processing performed by the accelerator 112 of the first embodiment.
[0135] When the accelerator 112 receives a decompression instruction from the processor 111 , it starts the decompression process.
[0136] The accelerator 112 acquires the compressed data from the memory 113 based on the arrangement location of the compressed data in the memory 113 (step S401 ).
[0137] Next, the accelerator 112 applies a decompression algorithm to the compressed data, thereby decompressing the compressed data (step S402 ).
[0138] Next, the accelerator 112 arranges the data included in the decompressed data string in the memory 113 based on the arrangement location (address) of the data string and the information indicating the arrangement location of each data in the data string (step S403 ).
[0139] For example, when the information indicating the arrangement location of each data in the data string is a bitmap, the accelerator 112 arranges the data included in the data string at the position corresponding to the bit set to 1. In addition, the data is arranged in the order of the data included in the data string.
[0140] Next, the accelerator 112 sends a completion notification to the processor 111 (step S404 ). Thereafter, the accelerator 112 ends the decompression process.
[0141] According to the first embodiment, the storage device 100 can reduce the processing load of the processor 111 by causing the accelerator 112 to perform the adjustment processing of data alignment, compression processing, and decompression processing. In addition, the processing units of the compression processing and the deduplication processing can be adjusted to the processing units suitable for each processing, so that the data reduction effect can be improved.
[0142] [Example 2]
[0143] In the second embodiment, the storage device 100 compresses a plurality of data strings at once, which is different from the first embodiment. The second embodiment will be described below, focusing on the differences from the first embodiment.
[0144] First, use Fig.10 and Fig.11 An overview of processing executed by the storage device 100 according to the first embodiment will be described. Fig.10 and Fig.11 This is a diagram for explaining an overview of processing executed by the storage device 100 according to the second embodiment.
[0145] First, use Fig.10 The outline of the write process executed by the storage device 100 of the first embodiment will be described. Here, the process will be described when the host 200 writes data strings 1000, 1010, and 1020 to different volumes. The sizes of the data strings 1000, 1010, and 1020 are the same.
[0146] First, the data strings 1000, 1010, and 1020 written by the host 200 are stored in the memory 113. After storing the data strings 1000, 1010, and 1020 in the memory 113, the processor 111 performs deduplication processing on each of the data strings 1000, 1010, and 1020. Fig.10 In the figure, data string 1000 retains data A, data string 1010 retains data B and C, and data string 1020 retains data D.
[0147] The processor 111 selects data of the amount of the compressed processing unit from the data string stored in the memory 113. Here, the size of the amount of data is 4 as the compressed processing unit. Therefore, data A, B, C, and D are selected. The processor 111 sends a compression instruction of data A, B, C, and D to the accelerator 112.
[0148] The accelerator 112 generates a data string ABCD according to the data A, B, C, and D, and generates compressed data abcd by compressing the data string. The accelerator 112 stores the compressed data abcd in the memory 113. Then, the processor 111 stores the compressed data abcd in the memory 113 in the storage medium 102. The compressed data is stored in the storage medium 102 without gaps.
[0149] When a portion of the data string is updated by the host 200, in the compressed data storage method of this embodiment, the original compressed data is not deleted, but the compressed data obtained by compressing the data string including the updated data is newly stored in the storage medium 102. At the same time, the physical address associated with the logical address is also updated. Data that is no longer associated with the logical address is called garbage.
[0150] Garbage is generated in large quantities as data strings are written to a volume. Therefore, the storage device 100 of the second embodiment deletes the garbage by performing a process called garbage collection. Furthermore, after the garbage is deleted, the storage device 100 reconfigures the compressed data of the storage medium 102. Thus, the storage area provided by the storage medium 102 to the volume used by the host 200 is ensured.
[0151] The above is an overview of the write process.
[0152] Next, use Fig.11 The outline of garbage collection performed by the storage device 100 of the first embodiment will be described. Here, the processing will be described when the host 200 writes a data string including update data C' corresponding to data C to the storage device 100. At this time, data C of the compressed data abcd becomes garbage.
[0153] The processor 111 reads the compressed data abcd from the storage medium 102 and stores it in the memory 113. The processor 111 sends a decompression instruction to the accelerator 112. The accelerator 112 decompresses the compressed data abcd and stores the data string ABCD in the memory 113.
[0154] The processor 111 sends a compression instruction for the data A, B, and D except the data C to the accelerator 112. The accelerator 112 generates a data string ABC based on the data A, B, and D, and generates compressed data abd by applying a compression algorithm. The accelerator 112 stores the compressed data abd in the memory 113. The processor 111 writes the compressed data abd at a position next to the end of the storage area for storing compressed data of the storage medium 102.
[0155] The above is an overview of garbage collection. As described above, the accelerator 112 re-compresses the data after eliminating garbage from the data string. The processor 111 does not need to directly process the data string, so the load can be reduced.
[0156] The hardware configuration of the storage device 100 of the second embodiment is the same as that of the first embodiment. In the second embodiment, the management information stored in the storage device 100 is different.
[0157] Fig.12 This is a diagram showing an example of compressed data management information 1200 according to the second embodiment.
[0158] The compressed data management information 1200 is information for managing compressed data. The compressed data management information 1200 stores entries consisting of a block number 1201, a physical address 1202, and a data size 1203. One entry corresponds to one compressed data.
[0159] The block number 1201 is a field for storing the block number for identifying the compressed data. In the second embodiment, since the data of a plurality of data strings are compressed at once, the logical addresses of the compressed data are not continuous. Therefore, the block number is used to identify the compressed data instead of the logical address.
[0160] The physical address 1202 and the data size 1203 are the same fields as the physical address 302 and the data size 303 .
[0161] Fig.13 This is a diagram showing an example of host data management information 1300 according to the second embodiment.
[0162] Host data management information 1300 is information for managing data deleted from a data string by deduplication processing. Host data management information 1300 stores entries consisting of a logical address 1301, a block number 1302, and an offset 1303. One entry corresponds to one piece of data deleted by deduplication processing.
[0163] The logical address 1301 is the same field as the logical address 311 .
[0164] The block number 1302 is a field for storing a block number assigned to compressed data obtained by compressing a data string including data.
[0165] The offset is a field storing a value indicating the position of data in the data string before compression.
[0166] Fig.14A and Fig. 14B This is a flowchart for explaining an example of data reduction processing executed by the processor 111 of the second embodiment.
[0167] The processor 111 performs data reduction processing when the write processing is executed or periodically. The write processing is executed when the storage device 100 receives a data string from the host 200. Here, it is assumed that the data reduction processing is executed when the write processing is executed. In addition, during the write processing, the storage device 100 stores the data string in the memory 113 and sends a completion notification of the write processing to the host 200.
[0168] The processor 111 selects a target data string from the data strings stored in the memory 113 (step S501).
[0169] The processor 111 divides the data string based on the processing unit of eliminating duplication (step S502).
[0170] Next, the processor 111 selects target data from the divided data (step S503).
[0171] Next, the processor 111 searches for data that matches the target data (step S504). The method of searching for data is the same as that of step S103.
[0172] The data to be compared are data stored in the memory 113 and the storage medium 102 .
[0173] Next, the processor 111 determines whether there is data that matches the target data based on the search result (step S505 ).
[0174] If there is data that matches the target data, the processor 111 registers information about the target data in the host data management information 1300 (step S506 ). Thereafter, the processor 111 proceeds to step S508 .
[0175] Specifically, the processor 111 adds an entry to the host data management information 1300, and stores the logical address of the target data in the logical address 1301 of the added entry. The block number of the compressed data including the retrieved data and the offset of the data are stored in the block number 1302 and the offset 1303 of the added entry.
[0176] If there is no data that matches the target data, the processor 111 registers the target data as compression target data (step S507 ). Thereafter, the processor proceeds to step S508 .
[0177] Specifically, the processor 111 registers data associating the address of the target data string in the memory 113 and the position (offset) of the data in the target data string in the list.
[0178] In step S508, the processor 111 determines whether the processing of all data of the target data string has been completed (step S508).
[0179] When it is determined that the processing of all the data in the target data string has not been completed, the processor 111 returns to step S503 and executes the same processing.
[0180] When it is determined that the processing of all the data in the target data string has been completed, the processor 111 determines whether the data amount of the compression target data is equal to or larger than the threshold value (step S509).
[0181] Specifically, the processor 111 calculates the amount of data to be compressed based on the number of data registered in the list and the processing unit of deduplication. The processor 111 determines whether the amount of data is greater than a threshold. Here, the threshold is the processing unit of compression.
[0182] When it is determined that the data size of the compression target data is smaller than the threshold value, the processor 111 returns to step S501 and executes the same process.
[0183] When it is determined that the amount of the compression target data is greater than the threshold, the processor 111 sends a compression instruction including a list to the accelerator 112 (step S510). The processor 111 enters a waiting state until receiving a completion notification from the accelerator 112. In addition, the completion notification includes the storage location (address) of the compressed data in the memory 113, the offset of each data in the data string before compression, and the data size of the compressed data.
[0184] When receiving the completion notification from the accelerator 112, the processor 111 determines the storage location of the compressed data in the storage medium 102 (step S511). Specifically, the following processing is performed.
[0185] (S511-1) The processor 111 refers to the physical address 1202 of the compressed data management information 1200 to identify the end position (address) of the storage area storing compressed data in the storage medium 102. The processor 111 determines the address of the storage area storing compressed data based on the address.
[0186] (S511-2) The processor 111 adds an entry to the compressed data management information 1200 and stores the block number in the block number 1201. The processor 111 stores the determined address in the physical address 1202 of the added entry. In addition, the processor 111 stores the notified data size of the compressed data in the data size 1203 of the entry.
[0187] The above is the description of the processing of step S511.
[0188] Next, the processor 111 updates the host data management information 1300 (step S512). Thereafter, the processor 111 ends the data reduction process. Specifically, the following process is performed.
[0189] (S512-1) The processor 111 selects target data from the compression target data registered in the list.
[0190] (S512-2) The processor 111 searches for an entry corresponding to the target data by referring to the host data management information 1300. Specifically, the processor 111 searches for an entry of the address storing the target data in the logical address 1301.
[0191] (S512-3) When the processor 111 determines that there is no entry corresponding to the target data in the host data management information 1300, that is, when the target data is data written to the volume for the first time, the processor 111 adds an entry to the host data management information 1300. The processor 111 stores the logical address of the target data in the logical address 1301 of the added entry. In addition, the processor 111 stores the block number of the compressed data in the block number 1302 of the added entry, and stores the offset of the target data in the offset 1303.
[0192] (S512-4) When the processor 111 determines that there is an entry corresponding to the target data in the host data management information 1300, that is, when updating the data, the processor 111 saves the block number of the compressed data in the block number 1302 of the retrieved entry, and saves the offset of the target data in the offset 1303 of the entry.
[0193] (S512-5) The processor 111 determines whether the processing of all compression target data has been completed. If the processing of all compression target data has not been completed, the processor 111 returns to S512-1 and performs the same processing. If the processing of all compression target data has been completed, the processor 111 ends the processing of step S512.
[0194] The above is an explanation of the processing of step S512. The result of the processing of step S512 is that the block number 1302 and the offset 1303 of the updated data are updated. At the same time, the physical address that is released from the association with the logical address is called garbage, and the physical address that is not garbage is called valid. In garbage collection, the compressed object data stored in the physical address in the valid state is read and stored in the storage medium 102 in a forward-aligned manner. In this way, an unused storage area of the storage medium 102 can be ensured.
[0195] After executing the data reduction process, the processor 111 stores the compressed data at a determined address of the storage medium 102 at an arbitrary timing. For example, the data reduction process and the compressed data storage process may be continuously performed, or the compressed data storage process may be periodically performed. When the compressed data storage process is periodically performed, the processor 111 retrieves the compressed data stored in the memory 113 and stores the retrieved compressed data in the storage medium 102.
[0196] Fig.15 This is a flowchart illustrating an example of compression processing performed by the accelerator 112 of the second embodiment.
[0197] When receiving a compression instruction from the processor 111 , the accelerator 112 starts the compression process.
[0198] The accelerator 112 acquires compression target data from the memory 113 based on the list notified from the processor 111 (step S601 ).
[0199] The accelerator 112 generates a data string based on the compression target data (step S602). At this time, the accelerator 112 generates data indicating the position (offset) of each compression target data in the data string.
[0200] Next, the accelerator 112 applies a compression algorithm to the generated data string, thereby generating compressed data (step S603).
[0201] Next, the accelerator 112 stores the compressed data in the memory 113 (step S604 ).
[0202] Next, the accelerator 112 sends a completion notification including the storage location (address) of the compressed data in the memory 113, the offset of each data in the data string before compression, and the data size of the compressed data to the processor 111 (step S605). Thereafter, the accelerator 112 ends the compression process.
[0203] The accelerator 112 can reduce the processing load of the processor 111 by compressing the data string.
[0204] The data reading process of the second embodiment is the same as that of the first embodiment, so the detailed description is omitted.
[0205] Next, garbage collection in the second embodiment is described.
[0206] Fig.16 This is a flowchart illustrating an example of garbage collection performed by the processor 111 of the second embodiment.
[0207] Garbage collection can be performed when garbage is generated or periodically.
[0208] The processor 111 selects a destination physical address (step S701). In this embodiment, a junk physical address with a large amount of data is selected to reduce the amount of data to be saved again.
[0209] For example, when the physical address is transferred from the valid state to the garbage state, the processor 111 stores the data amount of the compressed data stored in the physical address transferred to the garbage state as the garbage amount. The processor 111 selects the physical address based on the garbage amount.
[0210] The processor 111 acquires the compressed data from the storage medium 102 based on the destination physical address and stores it in the memory 113 (step S702 ).
[0211] Processor 111 sends a decompression instruction including the address of the compressed data in memory 113 to accelerator 112 (step S703). Processor 111 enters a waiting state until receiving a completion notification from accelerator 112. The completion notification includes the storage location (address) of the data string in memory 113.
[0212] The processor 111 selects target data from the data constituting the decompressed data string (step S704). At this time, the processor 111 generates the same number of bit strings (bitmaps) as the number of data included in the decompressed data string.
[0213] The processor 111 determines whether the target data is in a valid state (step S705). That is, it determines whether the target data needs to be saved. The target data that is not in a valid state is determined not to need to be saved.
[0214] Specifically, the processor 111 refers to the compressed data management information 1200 to obtain the value of the block number 1201 of the entry corresponding to the destination physical address. The processor 111 refers to the host data management information 1300 to search for an entry that stores the obtained block number in the block number 1302 and stores the offset of the target data in the data string in the offset 1303. If at least one of the above entries exists, it is determined that the target data is in a valid state.
[0215] If it is determined that the target data is garbage, the processor 111 proceeds to step S707.
[0216] When it is determined that the target data is in a valid state, the processor 111 registers the target data as compression target data (step S706 ). Thereafter, the processor 111 proceeds to step S707 .
[0217] Specifically, the processor 111 sets 1 in the bit corresponding to the target data in the bitmap.
[0218] In step S707, the processor 111 determines whether the processing of all the data of the decompressed data string has been completed (step S707).
[0219] If it is determined that the processing of all the data in the decompressed data string has not been completed, the processor 111 returns to step S704 and executes the same processing.
[0220] When it is determined that the processing of all the data in the decompressed data string has been completed, the processor 111 sends a compression instruction including the storage location (address) of the data string in the memory 113 and information about the compressed object data to the accelerator 112 (step S708). The processor 111 shifts to a waiting state until receiving a completion notification from the accelerator 112. In addition, the completion notification includes the storage location (address) of the compressed data in the memory 113, the offset of each data in the data string before compression, and the data size of the compressed data.
[0221] When receiving the completion notification from the accelerator 112, the processor 111 determines the storage location of the compressed data in the storage medium 102 (step S709), and then terminates the garbage collection. Specifically, the following processing is performed.
[0222] (S709-1) The processor 111 adds an entry to the compressed data management information 1200, and stores the block number of the compressed data in the block number 1201 of the added entry. In addition, the processor 111 stores the data size of the compressed data in the data size 1203 of the added entry.
[0223] (S709-2) The processor 111 refers to the physical address 1202 of the compressed data management information 1200 to identify the end position (address) of the storage area storing the compressed data of the storage medium 102. The processor 111 determines the address of the storage area storing the compressed data based on the address. The processor 111 stores the determined address in the physical address 1202 of the added entry.
[0224] (S709-3) The processor 111 refers to the host data management information 1300 and searches for an entry storing the logical address of the compression target data in the logical address 1301. The processor 111 stores the block number of the compressed data in the block number 1302 of all the searched entries. The processor 111 stores the offset of each data in the data string before compression in the offset 1303 of each searched entry.
[0225] The above is the description of the process of step S709.
[0226] In addition, if the accelerator 112 can perform the compression process and the decompression process as a series of processes, it is not necessary to send the decompression instruction and the compression instruction one by one. That is, the accelerator 112 performs the processes of S601 to S605 after the processes of S401 and S402. In this case, the processor 111 performs the processes of S701, S702, S704 to S707, and then sends the process execution to the accelerator 112. In this way, the same result can be obtained.
[0227] The storage device 100 of the second embodiment can suppress the processing load of the processor 111 by causing the accelerator 112 to perform the adjustment processing of data alignment, compression processing, and decompression processing. In addition, the processing units of the compression processing and the deduplication processing can be adjusted to the processing units suitable for each processing, so that the data reduction effect can be improved.
[0228] In addition, the present invention is not limited to the above-mentioned embodiments, and includes various modified examples. In addition, for example, the above-mentioned embodiments are described in detail to explain the present invention in an easy-to-understand manner, and are not limited to all structures that must be described. In addition, for a part of the structure of each embodiment, other structures can be added, deleted, or replaced.
[0229] In addition, for each of the above-mentioned structures, functions, processing parts, processing units, etc., part or all of them can be implemented by hardware, for example, by designing in an integrated circuit, etc. In addition, the present invention can also be implemented by the program code of the software that realizes the functions of the embodiments. In this case, a storage medium recording the program code is provided to the computer, and the processor possessed by the computer reads the program code stored in the storage medium. In this case, the program code read from the storage medium itself realizes the functions of the above-mentioned embodiments, and the program code itself and the storage medium storing it constitute the present invention. As a storage medium for supplying such a program code, for example, a floppy disk, CD-ROM, DVD-ROM, hard disk, SSD (Solid State Drive), optical disk, magneto-optical disk, CD-R, magnetic tape, non-volatile memory card, ROM, etc. are used.
[0230] In addition, the program code that realizes the functions described in the present embodiment can be implemented using a wide range of programming or scripting languages such as assembler, C / C++, perl, Shell, PHP, Python, Java (registered trademark), etc.
[0231] Furthermore, the program code of the software that implements the functions of the embodiment can be published via the network and stored in a storage unit such as a computer's hard disk or memory or a storage medium such as a CD-RW or CD-R, and the processor of the computer reads and executes the program code stored in the storage unit or the storage medium.
[0232] In the above embodiments, the control lines and information lines are shown as necessary for explanation, and not all the control lines and information lines on the product are necessarily shown. All structures may also be connected to each other.
[0233] Description of Reference Numerals
[0234] 100 Storage Device
[0235] 101 Storage Controller
[0236] 102 Storage Media
[0237] 111 Processor
[0238] 112 Accelerator
[0239] 113 Memory
[0240] 114 Host Interface
[0241] 200 Hosts
[0242] 300 Compressed data management information
[0243] 310 Duplicate elimination data management information
[0244] 1200 Compressed data management information
[0245] 1300 Host data management information.
Claims
1. A storage device having a processor, an accelerator, a memory and a storage component, Features: The processor determines compression target data and sends a compression instruction including information about the compression target data to the accelerator, wherein the compression target data is data to be stored in the storage component among the data stored in the memory, The accelerator reads the plurality of continuous data from the memory and compresses the plurality of compression target data except for the non-compression target data among the plurality of data based on the information about the compression target data received from the processor to generate compressed data to be stored in the storage component. The processor further performs a duplication elimination process on a data string consisting of a plurality of data stored in the memory, The non-compression target data is data that is not a target to be stored in the storage component due to the deduplication process. the processor determines the compression target data from the data constituting the first data string based on the result of the duplication elimination processing on the first data string, and generates first position information indicating the position of the compression target data in the first data string as the information on the compression target data, When receiving the compression instruction, the accelerator reads the first data string from the memory and obtains the compression target data from the first data string based on the first position information.
2. The storage device according to claim 1, Features: storing deduplication data management information associating the data deleted by the deduplication processing with a reference destination referred to in order to obtain the data, the processor, upon receiving a read request, acquiring data deleted due to the deduplication processing from a reference destination based on the deduplication data management information and storing the data in the memory, wherein the data deleted due to the deduplication processing is data constituting a read data string that is the object of the read request, Acquire the compressed data generated from the data constituting the read data string from the storage component and store it in the memory, sending a decompression instruction including second position information to the accelerator, wherein the second position information indicates the position of each of a plurality of data constituting a data string obtained by decompressing the compressed data in the read data string, The accelerator, upon receiving the decompression instruction, generates a second data string by decompressing the acquired compressed data, and stores the plurality of data constituting the second data string in the memory respectively in a manner capable of constituting the read data string based on the second position information.
3. The storage device according to claim 1, Features: The processor determines the compression target data of each of the plurality of data strings based on the result of the duplication elimination processing on the plurality of data strings, and generates third position information indicating the position of the compression target data in each of the plurality of data strings as the information on the compression target data, When receiving the compression instruction, the accelerator reads the plurality of data strings from the memory, and acquires a predetermined number of the compression target data from the plurality of data strings based on the third position information.
4. The storage device according to claim 3, Features: storing first compressed data generated from a data string consisting of the first data, the processor, When the first data does not need to be stored due to writing a new data string, the first compressed data is read from the storage component and stored in the memory, and a decompression instruction of the first compressed data is sent to the accelerator. The accelerator generates a third data string by decompressing the first compressed data and stores the third data string in the memory, The processor sends a compression instruction including information about the first data to the accelerator, The accelerator obtains data other than the first data from the data constituting the third data string, and compresses a fourth data string constituted by the obtained data to generate second compressed data.
5. The storage device according to claim 3, Features: storing first compressed data generated from a data string consisting of the first data, The processor, when the first data does not need to be stored due to writing a new data string, reads the first compressed data from the storage component and stores it in the memory, and sends a recompression instruction including information about the first data to the accelerator, The accelerator, upon receiving the re-compression instruction, generates a third data string by decompressing the first compressed data and stores it in the memory, obtains data other than the first data from the data constituting the third data string, and compresses a fourth data string constituted by the obtained data to generate second compressed data.
6. A data processing method performed by a storage device, Features: The storage device has a processor, an accelerator, a memory and a storage component, The data processing method comprises: The processor determines a first step of compressing target data, wherein the compressed target data is data to be stored in the storage component among the data stored in the memory; In a second step, the processor sends a compression instruction including information about the compression target data to the accelerator; In a third step, the accelerator reads a plurality of continuous data from the memory, and compresses a plurality of compression object data except for non-compression object data in the plurality of data based on the information about the compression object data received from the processor, to generate compressed data to be stored in the storage component; and The processor performs a step of performing a duplication elimination process on a data string consisting of a plurality of data stored in the memory, The non-compression target data is data that is not a target to be stored in the storage component due to the deduplication process. Wherein, the first step comprises: The processor determines the compression target data from the data constituting the first data string based on the result of the duplication elimination processing on the first data string; and the processor generating first position information indicating the position of the compression target data in the first data string as the information on the compression target data, The third step includes the step of the accelerator reading the first data string from the memory and acquiring the compression target data from the first data string based on the first position information.
7. The data processing method according to claim 6, Features: The storage device stores deduplication data management information that associates data deleted by the deduplication processing with a reference destination that is referenced to obtain the data. The data processing method comprises: The processor, when receiving a read request, acquires data deleted due to the deduplication processing from a reference destination based on the deduplication data management information and stores the data in the memory, wherein the data deleted due to the deduplication processing is data constituting a read data string that is the subject of the read request; The processor acquires the compressed data generated from the data constituting the read data string from the storage component and stores it in the memory; The processor sends a decompression instruction including second position information to the accelerator, wherein the second position information indicates the position of each of a plurality of data constituting a data string obtained by decompressing the compressed data in the read data string; The accelerator generates a second data string by decompressing the obtained compressed data; and The accelerator stores a plurality of data constituting the second data string in the memory in such a manner that the read data string can be constructed based on the second position information.
8. The data processing method according to claim 6, Features: The first step includes: The processor determines the compression target data of each of the plurality of data strings based on the result of the duplication elimination process on the plurality of data strings; and the processor generating third position information indicating the position of the compression target data in each of the plurality of data strings as information on the compression target data, The third step includes the step of the accelerator reading the plurality of data strings from the memory and acquiring a prescribed number of the compression target data from the plurality of data strings based on the third position information.
9. The data processing method according to claim 8, Features: The storage device stores first compressed data generated from a data string consisting of first data, The data processing method comprises: The processor reads the first compressed data from the storage component and stores it in the memory when the first data does not need to be stored due to writing a new data string; The processor sends a decompression instruction of the first compressed data to the accelerator; The accelerator generates a third data string by decompressing the first compressed data and stores the third data string in the memory; The processor sends a compression instruction including information about the first data to the accelerator; and The accelerator acquires data other than the first data from the data constituting the third data string, and compresses a fourth data string constituted by the acquired data to generate second compressed data.
10. The data processing method according to claim 8, Features: The storage device stores first compressed data generated from a data string consisting of first data, The data processing method comprises: The processor reads the first compressed data from the storage component and stores it in the memory when the first data does not need to be stored due to writing a new data string; The processor sends a recompression instruction including information about the first data to the accelerator; The accelerator, upon receiving the recompression instruction, generates a third data string by decompressing the first compressed data and stores the third data string in the memory; and The accelerator acquires data other than the first data from the data constituting the third data string, and compresses a fourth data string constituted by the acquired data to generate second compressed data.
