Storage device, method of operating storage device, and method of operating storage server

Abstract

Disclosed are an operating method of a storage device, an operating method of a storage server, and a storage device. The operating method of the storage device includes receiving a write request including an object identifier and data from an external device, performing a hash operation on the data to generate a hash value, determining whether an entry associated with the hash value in a table is empty, when it is determined that the entry is empty, storing the data in a region of the storage device corresponding to a physical address and updating the entry to include the physical address and a reference count, and when it is determined that the entry is not empty, increasing the reference count included in the entry without performing a storage operation associated with the data, and returning an error message to the external device when the entry associated with the hash value does not exist in the table.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to Korean Patent Application No. 10-2020-0164583, filed on November 30, 2020, with the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The embodiments of this disclosure described herein relate to a server system, and more specifically, to a storage device with deduplication capabilities, a method of operating the storage device, and a method of operating the storage server. Background Technology

[0004] A server system may include multiple computing servers connected via a network and providing various services. A server system may also include storage servers that store and manage large amounts of data. Users or objects may reference specific data in the storage server multiple times. Each reference may result in the creation and storage of duplicate or redundant data within the storage server. As the number of these references increases, the number of times duplicate data is stored also increases. This increase in the number of times duplicate data is stored reduces the amount of available storage space in the storage devices included in the storage server.

[0005] Deduplication techniques can be implemented to eliminate duplicate or redundant data within storage servers. However, current deduplication technologies require high-capacity storage, which increases manufacturing costs. Summary of the Invention

[0006] At least one embodiment of this disclosure provides a storage device with improved performance and low cost that supports deduplication, a method of operating the storage device, and a method of operating a storage server.

[0007] According to an embodiment, a method of operating a storage device includes: receiving a write request from an external device, including an object identifier and data; performing a hash operation on the data to generate a hash value; determining whether an entry in a table associated with the hash value is empty; when the entry is determined to be empty, storing the data in a region of the storage device corresponding to a physical address, and updating the entry to include the physical address and a reference count; and when the entry is determined to be non-empty, increasing the reference count included in the entry without performing a storage operation associated with the data, and returning an error message to the external device when no entry associated with the hash value exists in the table.

[0008] According to an embodiment, an operation method of a storage server including a plurality of storage devices and a storage node configured to manage the plurality of storage devices includes: receiving a first write request including a first object identifier and first data from an external client server at a storage node; performing a hash operation on the first data at the storage node to generate a first hash value; determining a first hash range including the first hash value from a plurality of hash ranges respectively allocated to the plurality of storage devices at the storage node; sending a second write request including the first object identifier, the first data, and the first hash value to a first storage device corresponding to the first hash range among the plurality of storage devices at the storage node; performing the hash operation on the first data included in the second write request at the first storage device to generate a second hash value; when the first hash value matches the second hash value and a first entry corresponding to the first hash value exists in a first table, incrementing a first reference count of the first entry at the first storage device without performing a storage operation associated with the first data; and sending a completion response to the storage node at the first storage device.

[0009] According to an embodiment, a storage device includes: a non-volatile memory; and a storage controller configured to control the non-volatile memory. The storage controller includes: a memory configured to store a first table; a hash module configured to perform a hash operation on first data received from an external device to generate a first hash value; and a deduplication manager configured to search the first table for a first entry corresponding to the first hash value, selectively storing the first data in the non-volatile memory based on the search results. When the first entry exists in the first table, the deduplication manager increments a first reference count of the first entry without storing the first data in the non-volatile memory. When the first entry does not exist in the first table, the deduplication manager stores the first data in a region corresponding to a first physical address in the non-volatile memory and adds a first entry including the first hash value, the first physical address, and the first reference count to the first table. Attached Figure Description

[0010] This disclosure will become clear from a detailed description of the embodiments thereof with reference to the accompanying drawings.

[0011] Figure 1 This is a block diagram illustrating a server system according to an embodiment of the present disclosure.

[0012] Figure 2 This illustrates an embodiment according to the present disclosure. Figure 1 A block diagram of the storage nodes.

[0013] Figure 3 This illustrates an embodiment according to the present disclosure. Figure 1A block diagram of one of a plurality of storage devices.

[0014] Figure 4 This illustrates an embodiment according to the present disclosure. Figure 1 The flowchart shows the operation of the storage server.

[0015] Figure 5 This is a diagram used to describe a hash-physical table managed by a storage device according to embodiments of the present disclosure.

[0016] Figure 6 This illustrates an embodiment according to the present disclosure. Figure 1 The flowchart of the server system operation.

[0017] Figure 7 and Figure 8 It is used to describe according to Figure 6 The flowchart illustrates the operation.

[0018] Figure 9 This illustrates an embodiment according to the present disclosure. Figure 1 The flowchart of the server system operation.

[0019] Figure 10 It is used to describe according to Figure 9 A flowchart illustrating the read operation.

[0020] Figure 11 This illustrates an embodiment according to the present disclosure. Figure 1 The flowchart of the server system operation.

[0021] Figure 12 It is used to describe according to Figure 11 The flowchart illustrates the deletion operation.

[0022] Figure 13 This illustrates an embodiment according to the present disclosure. Figure 1 A block diagram illustrating the operation of the storage device.

[0023] Figure 14 This illustrates an embodiment according to the present disclosure. Figure 1 The flowchart of the server system operation.

[0024] Figure 15 It is used to describe according to Figure 14 The flowchart illustrates the operation.

[0025] Figure 16 This illustrates an embodiment according to the present disclosure. Figure 1 The flowchart of the server system operation.

[0026] Figure 17 This illustrates an embodiment according to the present disclosure. Figure 1The flowchart of the server system operation.

[0027] Figure 18 and Figure 19 It shows a reference. Figure 16 and Figure 17 A diagram illustrating an example of data segmentation.

[0028] Figure 20 This is a diagram illustrating a server system according to an embodiment of the present disclosure.

[0029] Figure 21 It is used to describe Figure 20 A diagram illustrating the hash range managed at each storage node of the server system.

[0030] Figure 22 This illustrates an embodiment according to the present disclosure. Figure 20 The flowchart of the server system operation.

[0031] Figure 23 This illustrates an embodiment according to the present disclosure. Figure 20 The flowchart of the server system operation.

[0032] Figure 24 This is a block diagram illustrating a data center employing a storage device according to an embodiment of the present disclosure. Detailed Implementation

[0033] The embodiments of this disclosure are described below in detail and clearly so that those skilled in the art can implement this disclosure.

[0034] Hereinafter, for ease of description, specific embodiments will be distinguished and described, but the scope of the invention is not limited thereto. For example, it should be understood that various embodiments can be combined, or a portion of one embodiment can be combined with a portion of another embodiment.

[0035] Figure 1 This is a block diagram illustrating a server system according to an embodiment of the present disclosure. (Refer to...) Figure 1 The server system 100 includes a client server 101 and a storage server 102. The server system 100 can be a data center or data storage center, performing maintenance on various types of data and providing various services for the data. The server system 100 can be a system used for search engines or database management, or a computing system used in various organizations. The server system 100 can be a storage system providing cloud services or on-premises services.

[0036] Client server 101 can refer to a user, user terminal, or user computing system that uses various services related to various data. Client server 101 can store data in storage server 102, or can read data stored in storage server 102.

[0037] Upon request from client server 101, storage server 102 may store data or provide stored data to client server 101. In this embodiment, client server 101 and storage server 102 communicate with each other via a computer network (not shown). For example, the computer network may be the Internet. For example, client server 101 and storage server 102 may communicate with each other wirelessly or via a wired connection.

[0038] Storage server 102 includes storage node 110 and multiple storage devices 120 and 130. Storage node 110 can be configured to manage the storage devices 120 and 130 included in storage server 102. Under the control of storage node 110, each of the multiple storage devices 120 and 130 can store data or output data stored therein. Each of the multiple storage devices 120 and 130 may include a high-capacity storage medium such as a solid-state drive (SSD), but this disclosure is not limited thereto. Storage node 110 and each of the multiple storage devices 120 and 130 can communicate with each other through an object-based interface. For example, unlike a block-based interface, an object-based interface may refer to an interface that manages data based on information about objects. In embodiments, storage node 110 may refer to a server controller included in storage server 102, but this disclosure is not limited thereto.

[0039] Storage server 102 can provide deduplication functionality for data. For example, multiple objects can be driven on client server 101. In an embodiment, an object can refer to information used to manage data stored in storage server 102. For example, an object can refer to a file. In an embodiment, multiple objects can be generated on different virtual machines or different applications, and multiple objects can each correspond to different files or different groups of files. Conversely, a portion of multiple objects can correspond to the same file.

[0040] In embodiments where both the first object and the second object correspond to the second file, the storage server 102 can store the first file corresponding to the first object and the second file corresponding to the second object. Under these conditions, if the first file and the second file are identical, two identical files can be stored, resulting in a waste of storage space for the storage server 102. Therefore, regarding multiple objects corresponding to the same file, the storage server 102 according to embodiments of this disclosure can prevent the waste of storage space by storing only one of the identical files that correspond to each of the multiple objects. This technical configuration is called a "deduplication function".

[0041] Storage node 110 includes an object-hash table O2H. The object-hash table O2H includes mapping information between object identifiers and hash values. Each of the plurality of storage devices 120 and 130 includes a hash-physical address table H2P. The hash-physical address table H2P may include information about the physical address assigned to the hash value and information about a reference count corresponding to the hash value. Storage server 102 can provide the above-described deduplication functionality based on the object-hash table O2H and hash-physical address table H2P. The deduplication functionality according to embodiments of the present disclosure will be described more fully with reference to the following drawings.

[0042] Figure 2 It is shown Figure 1 A block diagram of the storage nodes. (Refer to...) Figure 1 and Figure 2 Storage node 110 includes processor 111, memory 112, hash module 113 (e.g., logic circuitry), error processor 114 (e.g., error detection and correction circuitry), network interface controller (NIC) 115, host interface (I / F) circuitry 116, and object-hash table O2H.

[0043] The processor 111 can control the overall operation of the storage node 110. The memory 112 can be used as a buffer memory, cache memory, or working memory for the storage node 110.

[0044] Hash module 113 can be configured to perform a hash operation on data to generate a hash value and output a hash value corresponding to the data. For example, when a write request for first data corresponding to a first object is received from client server 101, hash module 113 can perform a hash operation on the first data to output a first hash value. Alternatively, when a write request for second data corresponding to a second object is received from client server 101, hash module 113 can perform a hash operation on the second data to output a second hash value. According to embodiments of this disclosure, the first hash value and the second hash value being the same can mean that the first data and the second data are the same. In other words, the parameters of the hash operation of hash module 113 can be set such that the same hash value is generated for the same data.

[0045] Error handler 114 can be configured to handle various errors that occur during the operation of storage node 110. The operation of error handler 114 will be described more fully with reference to the following figures.

[0046] Network interface controller 115 can provide communication with client server 101. In embodiments where client server 101 and storage server 102 communicate with each other via an Ethernet network, network interface controller 115 can be configured to generate or process communication requests or packets based on the TCP / IP protocol, but this disclosure is not limited thereto. For example, network interface controller 115 can operate based on various communication protocols.

[0047] The host interface circuit 116 can provide communication with multiple storage devices 120 and 130. In embodiments, the host interface circuit 116 can communicate with multiple storage devices 120 and 130 based on at least one of various interfaces, such as ATA (Advanced Technology Attachment) interface, SATA (Serial ATA) interface, e-SATA (External SATA) interface, SCSI (Small Computer Small Interface), SAS (Serial Attached SCSI) interface, PCI (Peripheral Component Interconnect) interface, PCIe (PCI Fast) interface, NVMe (NVM Fast) interface, IEEE 1394 interface, USB (Universal Serial Bus) interface, SD (Secure Digital) card interface, MMC (Multimedia Card) interface, eMMC (Embedded Multimedia Card) interface, UFS (Universal Flash Memory) interface, eUFS (Embedded Universal Flash Memory) interface, CF (Compact Flash Memory) card interface, and KV (Key-Value) interface.

[0048] The object-hash table O2H includes mapping information between object identifiers and hash values. For example, as described above, when the hash value of the first data corresponding to the first object is a first hash value, the object-hash table O2H includes mapping information between the first object identifier and the first hash value of the first object. In embodiments, information managed by using the object-hash table O2H can be updated during write or erase operations of the storage server 102. The object-hash table O2H can be stored in the memory 112 or in separate external memory. For example, if a new object is being written, a new entry can be generated in the object-hash table O2H. For example, if an existing object is being deleted, an existing entry for the existing object can be deleted from the object-hash table O2H.

[0049] Figure 3 It is shown Figure 1 A block diagram of one of a plurality of storage devices. For ease of description, reference will be made to... Figure 3 This disclosure describes a storage device 120, but is not limited thereto. For example, other storage devices may also have the same... Figure 3 The structure shown is similar to the one shown.

[0050] Reference Figure 1 and Figure 3 The storage device 120 includes a storage controller 121, non-volatile memory (NVM) 122, and buffer memory 123. In response to a request received from storage node 110, the storage controller 121 may store data in the non-volatile memory 122 or output the data stored therein. For example, the request may be a read request or a write request.

[0051] The storage controller 121 includes a processor 121a, a memory 121b, a hash module 121c, a deduplication manager 121d, a flash translation layer (FTL) 121e, an Advanced Encryption Standard (AES) engine 121f, an ECC engine 121h, a host interface (I / F) circuit 121i, a memory interface (I / F) circuit 121j, and a hash-physical address table H2P (hereinafter referred to as the "hash-physical table").

[0052] Processor 121a can control the overall operation of storage controller 121. Memory 121b can be used as a buffer memory, high-speed cache memory, or working memory for storage controller 121. In an embodiment, hash module 121c is configured to calculate the hash value of the data. In an embodiment, Figure 3 The storage controller 121 includes a hash module 121c and Figure 2 The hash module 113 included in storage node 110 uses the same hash function. That is, hash modules 113 and 121c calculate the same hash value for the same data.

[0053] The deduplication manager 121d can be configured to perform deduplication on data stored in storage device 120. For example, the deduplication manager 121d searches for data that are referenced by different objects but have the same value based on a hash-physical table (H2P), and stores only one of the found data in non-volatile memory 122. The operation of the deduplication manager 121d will be described more fully with reference to the following figures.

[0054] The flash conversion layer 121e can perform various maintenance operations on the non-volatile memory 122. In embodiments, maintenance operations on the non-volatile memory 122 may include wear leveling operations, garbage collection operations, bad block management operations, etc.

[0055] In an embodiment, the flash translation layer 121e according to an embodiment of the present disclosure does not perform legacy mapping operations between logical addresses and physical addresses.

[0056] For example, the physical address of stored data can be managed through a hash-physical table (H2P) and a deduplication manager (121d). That is, the deduplication manager (121d) can manage the physical address of data based on the hash value, which is the result of performing a hash operation on the data, rather than based on the logical address of the data. In this case, a separate mapping between logical and physical addresses via a flash translation layer (121e) is unnecessary.

[0057] In embodiments, the hash value according to embodiments of this disclosure corresponds to information different from the logical address. For example, the logical address may indicate location information for logically managing the storage space of storage devices 120 and 130. That is, the logical address is information indicating the logical storage location of data. Conversely, the hash value is a value obtained by performing a hash operation on the data itself, not a value indicating the logical storage location of the data. In the case where first data and second data with the same value have different logical storage locations, the logical addresses of the first data and the second data may be different. Conversely, even if first data and second data with the same value have different logical storage locations, the hash value as the result of performing a hash operation on the first data and the hash value as the result of performing a hash operation on the second data are the same. That is, the hash value used in this disclosure corresponds to information whose characteristics are different from those of the logical address.

[0058] The AES engine 121f can perform at least one of encryption and decryption operations on the data input to the storage controller 121 using a symmetric key algorithm.

[0059] The ECC engine 121h can perform error detection and correction functions on read data read from the non-volatile memory 122. For example, the ECC engine 121h can generate parity bits for the written data to be stored in the non-volatile memory 122, and the generated parity bits can be stored in the non-volatile memory 122 along with the written data. When reading data from the non-volatile memory 122, the ECC engine 121h corrects errors in the read data by using the parity bits read from the non-volatile memory 122 along with the read data, and can output the error-corrected read data.

[0060] The host interface circuit 121i can communicate with the storage node 110. In embodiments, the host interface circuit 121i can be implemented based on at least one of the following protocols for the interface: ATA (Advanced Technology Attachment) interface, SATA (Serial ATA) interface, e-SATA (External SATA) interface, SCSI (Small Computer Small Interface) interface, SAS (Serial Attached SCSI) interface, PCI (Peripheral Component Interconnect) interface, PCIe (PCI Fast) interface, NVMe (NVM Fast) interface, IEEE 1394 interface, USB (Universal Serial Bus) interface, SD (Secure Digital) card interface, MMC (Multimedia Card) interface, eMMC (Embedded Multimedia Card) interface, UFS (Universal Flash Storage) interface, eUFS (Embedded Universal Flash Storage) interface, CF (Compact Flash Memory) card interface, and KV (Key-Value) interface.

[0061] The memory interface circuit 121j provides communication with multiple non-volatile memories 122. The memory interface circuit 121j can operate based on at least one of the following protocols for the interface: switching interface and open NAND flash interface (ONFI).

[0062] Buffer memory 123 may be configured to store various information required for the operation of storage device 120. For example, buffer memory 123 may store information about hash-physical table H2P. Alternatively, buffer memory 123 may be configured to store software components such as deduplication manager 121d and flash conversion layer 121e. Software components stored in buffer memory 123 may be loaded onto memory 121b and executed by processor 121a.

[0063] Figure 4 This illustrates an embodiment according to the present disclosure. Figure 1 A flowchart illustrating the operation of the storage server. (Refer to...) Figure 1 and Figure 4In operation S11, storage node 110 receives information about the size of the first storage device 120. This information may refer to information about the available storage space of the first storage device 120. In operation S12, storage node 110 receives information about the size of the second storage device 130. This information may refer to information about the available storage space of the second storage device 130.

[0064] In operation S13, storage node 110 allocates hash ranges to the first storage device 120 and the second storage device 130, respectively. For example, storage node 110 may allocate a first hash range having a range corresponding to the first device size information to the first storage device 120, and may allocate a second hash range having a range corresponding to the second device size information to the second storage device 130.

[0065] In operation S14, storage node 110 sends information about the first hash range to first storage device 120. In operation S15, first storage device 120 generates a first hash-physical table based on the first hash range information.

[0066] In operation S16, storage node 110 sends information about the second hash range to second storage device 130. In operation S17, second storage device 130 generates a second hash-physical table based on the second hash range information.

[0067] In this embodiment, the first hash range and the second hash range do not overlap. That is, storage node 110 can manage data such that data having hash values ​​included in the first hash range is stored in the first storage device 120, and data having hash values ​​included in the second hash range is stored in the second storage device 130.

[0068] In the embodiments, the hash value according to this disclosure corresponds to information having characteristics different from those of the logical address described above; therefore, additional descriptions will be omitted to avoid redundancy.

[0069] Figure 5 This is a diagram used to describe a hash-physical table managed by a storage device. Hereinafter, for ease of description, embodiments of this disclosure will be described with reference to the communication between storage node 110 and the first storage device 120. However, this disclosure is not limited thereto. (Refer to...) Figure 1 and Figure 5 The Hash-Physical Table (H2P) includes mapping information between multiple hash values ​​H1 to Hn and multiple physical addresses PA1 to PAn, as well as information about reference counts RC1 to RCn corresponding to the multiple hash values ​​H1 to Hn.

[0070] For example, if the hash value of the first data is a first hash value H1, the first data can be stored in the region corresponding to the first physical address PA1. In this case, the first reference count RC1 can indicate the number of objects referencing the first data corresponding to the first hash value H1. That is, the first reference count RC1 being "a" can mean either the number of objects referencing the first data corresponding to the first hash value H1 or the first data stored in the region of the first physical address PA1 being "a".

[0071] In this embodiment, if no data corresponding to the hash value is stored, the entry in the hash-physical table H2P corresponding to the hash value may be empty. For example, if the data corresponding to the third hash value H3 has not yet been stored in the storage device 120, the entry corresponding to the third hash value H3 may be empty.

[0072] In this embodiment, management is performed via the first storage device 120. Figure 5 In the case of the hash-physical table H2P shown, the multiple hash values ​​H1 to Hn included in the hash-physical table H2P can be included in the first hash range allocated through storage node 110.

[0073] Figure 6 This illustrates an embodiment according to the present disclosure. Figure 1 A flowchart of the operation of the server system is provided. For ease of discussion, embodiments of this disclosure will be described with reference to the client server 101, storage node 110, and first storage device 120 of the server system 100. However, this disclosure is not limited thereto. In the embodiments, reference will be made to... Figure 6 This describes the operation (i.e., write operation) in which data from a specific object is stored in a storage device.

[0074] Reference Figure 1 and Figure 6 In operation S101, the client server 101 sends a write request to the storage server 102. In this embodiment, the write request may be a write request PUT that includes an object identifier ID, a data length len, and a data DT.

[0075] In operation S102, storage node 110 calculates a hash value. For example, storage node 110 can calculate the hash value "H" by performing a hash operation on the data DT received from client server 101. In an embodiment, this can be done by referring to... Figure 2 The hash module 113 described performs the hash operation of the storage node 110.

[0076] In operation S103, storage node 110 transmits a write request to first storage device 120. In an embodiment, the write request transmitted to first storage device 120 may be a write request PUTa that includes object identifier ID, data length len, data DT, and hash value "H".

[0077] In an embodiment, even if not shown, storage node 110 can select the storage device to which the write request is to be transferred based on the calculated hash value "H". For example, as shown in reference... Figure 4 As described, storage node 110 can allocate hash ranges to each of a plurality of storage devices 120 and 130. Storage node 110 can transfer write requests to the storage device corresponding to a hash range that includes the calculated hash value "H". For example, if the hash value "H" is included in the first hash range allocated to the first storage device 120, the write request can be transferred to the first storage device 120; if the hash value "H" is included in the second hash range allocated to the second storage device 130, the write request can be transferred to the second storage device 130.

[0078] In operation S104, the first storage device 120 calculates the hash value "H" by performing a hash operation on the received data DT. In an embodiment, this can be achieved by referring to... Figure 3 The hash module 121c described performs the hash operation of the first storage device 120.

[0079] In operation S105, the first storage device 120 compares the calculated hash value "H" with the received hash value "H". A difference between the calculated hash value "H" and the received hash value "H" may indicate that an error is included in the received information. If the calculated hash value "H" and the received hash value "H" differ, in operation S106, the first storage device 120 may return an error message to the storage node 110.

[0080] In operation S107, storage node 110 performs error handling. In an embodiment, error handling in operation S107 includes re-hashing the data DT, resending the write request (e.g., PUTa[ID, len, DT, H]) to the first storage device 120, etc. Alternatively, error handling may include returning error information to the client server 101. This can be illustrated by referring to... Figure 2 The error handler 114 described herein performs the above error handling, but this disclosure is not limited thereto.

[0081] When the calculated hash value "H" matches the received hash value "H", in operation S108, the first storage device 120 determines whether the entry in the hash-physical table H2P corresponding to the received hash value "H" is empty.

[0082] An empty entry in the hash-physical table H2P corresponding to the received hash value "H" may mean that the data DT corresponding to the received hash value "H" is not stored in the first storage device 120. In the case that the entry in the hash-physical table H2P corresponding to the received hash value "H" is empty, in operation S109, the first storage device 120 allocates a physical address PA. For example, the allocation of physical address PA may include assigning a physical address to the hash value. In operation S110, the first storage device 120 writes the data DT to the region corresponding to the allocated physical address PA (i.e., the storage region of the non-volatile memory 122). In operation S111, the first storage device 120 updates the hash-physical table H2P. In operation S112, the first storage device 120 increments the relevant reference count RC.

[0083] For example, as described above, since the entry corresponding to the received hash value "H" is empty, it means that the data DT corresponding to the received hash value "H" is not stored in the first storage device 120, so the data DT is stored in the non-volatile memory 122. Therefore, the first storage device 120 can allocate a physical address PA and can store the data DT in the area corresponding to the allocated physical address PA. Then, the first storage device 120 can update the hash-physical table H2P such that the physical address corresponding to the received hash value "H" is set to the allocated physical address. For example, the entry in the hash-physical table H2P associated with the received hash value "H" can be updated to the allocated physical address. Then, since the number of objects referencing the data DT corresponding to the hash value "H" is "1", the reference count RC corresponding to the hash value "H" can be increased by "1". For example, if the entry is initially empty, its reference count RC can be increased from 0 to 1.

[0084] The fact that the entry in the hash-physical table H2P corresponding to the received hash value "H" is not empty indicates that the data DT corresponding to the received hash value "H" has been stored in the first storage device 120. When the entry in the hash-physical table H2P corresponding to the received hash value "H" is not empty, the first storage device 120 can omit operations S109 to S111. That is, because the data DT corresponding to the received hash value "H" has already been stored in the non-volatile memory 122, the first storage device 120 can omit the operation of storing the data DT. In other words, deduplication associated with the data DT can be performed.

[0085] If the entry in the hash-physical table H2P corresponding to the received hash value "H" is not empty, operation S112 is performed. For example, the data DT that has been stored corresponding to the received hash value "H" may refer to an additional object that has been stored as reference data DT. Therefore, the first storage device 120 can increase the reference count RC.

[0086] In operation S113, the first storage device 120 sends a completion message to the storage node 110. The completion message indicates to the storage node 110 that the data writing has been completed.

[0087] In operation S114, storage node 110 updates the object-hash table O2H. For example, storage node 110 may update the object-hash table O2H based on the mapping information between the object identifier ID and the hash value "H" received in operation S101. In an embodiment, the object-hash table O2H can be used to determine which hash value corresponds to the object identifier, and this operation can be used in read operations.

[0088] In operation S115, storage node 110 can send a success message to client server 101. The success message indicates whether the write operation was successfully completed.

[0089] Figure 7 and Figure 8 It is used to describe according to Figure 6 The flowchart below illustrates the operation. For the sake of brevity and ease of description, details describing the process are omitted. Figure 6 The flowchart write operation does not require components. In the embodiment, for the sake of brevity of the drawings and for ease of description, components related to... Figure 6 The components associated with operations S104 and S105 (i.e., error detection for the received request). However, this disclosure is not limited thereto.

[0090] Reference Figure 1 , Figure 6 and Figure 7 Client server 101 sends a first write request PUT1 to storage node 110. The first write request PUT1 may be a write request for first data DT1 corresponding to an object reference of the first object identifier ID1, and may include information about the first object identifier ID1 and the first data DT1.

[0091] The hash module 113 of storage node 110 calculates a first hash value H1 by performing a hash operation on the first data DT received in this way. The host interface circuit 116 of storage node 110 sends a first write request PUT1a, which includes the first hash value H1, to the first storage device 120. The first write request PUT1a may also include the first data DT and a first object identifier ID1.

[0092] In this embodiment, as described above, storage node 110 can select a storage device corresponding to a hash range including the first hash value H1 based on the first hash value H1 and the hash ranges allocated to the plurality of storage devices 120 and 130 respectively. Storage node 110 can send a first write request PUT1a including the first hash value H1 to the selected storage device.

[0093] The host interface circuit 121i of the first storage device 120 can receive a first write request PUT1a including a first hash value H1. The deduplication manager 121d of the first storage device 120 can search the hash-physical table H2P for the first hash value H1. Figure 7 In this embodiment, the entry corresponding to the first hash value H1 in the hash-physical table H2P can be empty. In this case, the deduplication manager 121d can allocate the first physical address PA1. In this embodiment, this can be achieved through the flash translation layer (FTL) 121e (see reference). Figure 3 ) Perform the operation of allocating physical addresses.

[0094] The memory interface circuit 121j of the first storage device 120 can send the program command CMD_P, the first physical address PA1, and the first data DT1 to at least one of a plurality of non-volatile memories 122. The first physical address PA1 may correspond to an assigned physical address. At least one of the plurality of non-volatile memories 122 can store the first data DT1 in the area corresponding to the first physical address PA1 in response to the program command CMD_P.

[0095] The deduplication manager 121d can update the hash-physical table H2P based on a first hash value H1 and a first physical address PA1. For example, as... Figure 7 As shown, the deduplication manager 121d can update the hash-physical table H2P to the first hash-physical table H2P-1 by adding an entry to the hash-physical table H2P that includes mapping information between the first hash value H1 and the first physical address PA1 and the first reference count RC1.

[0096] In an embodiment, storage node 110 can update the object-hash table O2H. For example, storage node 110 can add the mapping information between the first object identifier ID1 and the first hash value H1 to the object-hash table O2H. For example, after adding an entry including the first hash value H1 to the hash-physical table H2P, an entry can be added to the object-hash table O2H that includes the first object identifier ID1 and the first hash value H1.

[0097] Reference Figure 1 , Figure 6 and Figure 8Client server 101 sends a second write request PUT2 to storage node 110. The second write request PUT2 may be a write request for first data DT1 corresponding to an object reference of the second object identifier ID2, and may include information about the second object identifier ID2 and the first data DT1.

[0098] The hash module 113 of storage node 110 calculates a first hash value H1 by performing a hash operation on the received first data DT1. In an embodiment, the first data DT1 corresponding to the first object identifier ID1 can be compared with a reference... Figure 7 The first data DT1 corresponding to the first object identifier ID1 is the same. That is, objects corresponding to the first object identifier ID1 and the second object identifier ID2 can reference the same data, namely, the first data DT1. In this case, the hash value calculated by the hash module 113 is the same as the first hash value H1. The host interface circuit 116 of the storage node 110 sends a second write request PUT2a including the first hash value H1 to the first storage device 120.

[0099] The host interface circuit 121i of the first storage device 120 receives a second write request PUT2a including a first hash value H1. The deduplication manager 121d searches a first hash-physical table H2P-1 based on the received first hash value H1. For example, the first hash-physical table H2P-1 may include a mapping between the first hash value H1 and a first physical address PA1, as well as a first reference count RC1, such as a reference... Figure 7 The description.

[0100] In this scenario, the deduplication manager 121d determines that an entry corresponding to the first hash value H1 exists, and then generates the second hash-physical table H2P-2 by updating the first hash-physical table H2P-1, without performing any storage, programming, or writing operations to the first data DT1. For example, the deduplication manager 121d can generate the second hash-physical table H2P-2 in the first hash-physical table H2P-1 by incrementing the first reference count RC1 of the entry corresponding to the first hash value H1 by "1". Figure 8 In one embodiment, the first reference count RC1 can be updated to "2", which means that two objects (e.g., the object corresponding to the first object identifier ID1 and the object corresponding to the second object identifier ID2) reference the first data DT1 associated with the first hash value H1.

[0101] In other words, as referenced Figure 8 According to the description, the first storage device 120 may perform or omit the following operation: storing the received data in non-volatile memory 122 based on hash-physical table H2P.

[0102] In an embodiment, storage node 110 can update object-hash table O2H based on the mapping information between the second object identifier ID2 and the first hash value H1.

[0103] As described above, according to embodiments of this disclosure, storage server 102 can manage data based on the hash value of data DT. In this case, even if multiple objects reference the same data, only one piece of data is stored in storage server 102. Therefore, the utilization of storage space in non-volatile memory or storage devices can be improved.

[0104] Figure 9 It is shown Figure 1 A flowchart of the operation of the server system. In this embodiment, reference will be made to... Figure 9 Describes the operation (i.e., read operation) in which a specific object reads data from a storage device. (See also...) Figure 1 and Figure 9 In operation S201, client server 101 sends a read request GET to storage node 110 to perform a read operation. In this embodiment, the read request GET includes information about the object identifier ID.

[0105] In operation S202, storage node 110 searches the object-hash table O2H for the hash value "H" corresponding to the received object identifier ID. For example, as described above, the object-hash table O2H of storage node 110 can be used to manage the mapping information between object identifier IDs and hash values ​​"H". Storage node 110 can search the object-hash table O2H for the hash value "H" corresponding to the received object identifier ID.

[0106] In operation S203, storage node 110 sends a read request GETa, including the found hash value "H", to the first storage device 120. In this embodiment, storage node 110 selects the storage device to which the read request will be transmitted based on the found hash value "H", and this operation is similar to referencing... Figure 6 The description is omitted to avoid redundancy. A read request GETa may also include an object identifier (ID).

[0107] In operation S204, the first storage device 120 determines whether there is an entry in the hash-physical table H2P corresponding to the received hash value "H". The absence of an entry in the hash-physical table H2P corresponding to the received hash value "H" may mean that data corresponding to the hash value "H" is not stored in the non-volatile memory 122. If there is no entry in the hash-physical table H2P corresponding to the received hash value "H", in operation S205, the first storage device 120 may return an error to the storage node 110. For example, the first storage device 120 may send an error message to the storage node 110. In operation S206, the storage node 110 performs error handling. Error handling may include resending the read request GET or sending an error message to the client.

[0108] If an entry corresponding to the received hash value "H" exists in the hash-physical table H2P, in operation S207, the first storage device 120 reads the data DT from the non-volatile memory 122. For example, the first storage device 120 can search for the physical address PA corresponding to the hash value "H" in the hash-physical table H2P. The first storage device 120 can then read the data DT from the region corresponding to the found physical address PA.

[0109] In operation S208, the first storage device 120 performs a hash operation on the read data DT to calculate a hash value. In operation S209, the first storage device 120 compares the calculated hash value with the received hash value "H". A mismatch between the calculated hash value and the received hash value "H" may indicate that the requested data is different from the read data. In the event that the calculated hash value and the received hash value "H" do not match, in operation S210, the first storage device 120 may return an error to the storage node 110. For example, the first storage device 120 may send an error message to the storage node 110. In operation S211, the storage node 110 may perform error handling. In embodiments, even if not shown, if the calculated hash value is different from the received hash value "H", the first storage device 120 may re-perform a read operation on the requested data based on various read methods.

[0110] If the calculated hash value matches the received hash value "H", in operation S212, the first storage device 120 sends the data DT to the storage node 110. In operation S213, the storage node 110 sends the data DT to the client server 101.

[0111] Figure 10 It is used to describe according to Figure 9 The flowchart below illustrates the read operation. For the sake of brevity and ease of description, further details are omitted. Figure 9The flowchart read operation does not require any components. In the embodiment, for the sake of brevity of the drawings and for ease of description, components related to... Figure 9 The components associated with operations S204, S205, S209, and S210 (i.e., error detection). However, this disclosure is not limited thereto.

[0112] Reference Figure 1 , Figure 9 and Figure 10 Client server 101 sends a first read request GET1, including a first object identifier ID1, to storage node 110. The first read request GET1 may be a request to read first data DT1 corresponding to the object reference of the first object identifier ID1.

[0113] The hash module 113 of storage node 110 can search the object-hash table O2H for a hash value "H" corresponding to the received first object identifier ID1. For example, the object-hash table O2H may include information that the first object identifier ID1 and the second object identifier ID2 are mapped to the first hash value H1. In this case, the hash module 113 selects the first hash value H1 as the hash value corresponding to the first object identifier ID1 based on the object-hash table O2H. The host interface circuitry 116 of storage node 110 sends a first read request GET1a, which includes the first hash value H1, to the first storage device 120. The first read request GET1a may also include the first object identifier ID1.

[0114] The host interface circuit 121i of the first storage device 120 receives a first read request GET1a including a first hash value H1. The deduplication manager 121d can search in the second hash-physical table H2P-2 for a first physical address PA1 corresponding to the first hash value H1. Information about the first physical address PA1 can be transmitted to the memory interface circuit 121j of the first storage device 120.

[0115] The memory interface circuit 121j of the first storage device 120 sends the read command CMD_R and the first physical address PA1 to at least one associated non-volatile memory of the non-volatile memory 122. The at least one associated non-volatile memory of the non-volatile memory 122 can output the first data DT1 stored in the area of ​​the first physical address PA1.

[0116] The memory interface circuit 121j of the first storage device 120 can receive first data DT1 from at least one associated non-volatile memory of the non-volatile memory 122, and can provide the first data DT1 to the client server 101 through the host interface circuits 121i and 116.

[0117] Figure 11It is shown Figure 1 A flowchart of the operation of the server system. In this embodiment, reference will be made to... Figure 11 Describes the operation (i.e., deletion operation) of deleting data of a specific object from a storage device.

[0118] Reference Figure 1 and 11 In operation S301, client server 101 may send a delete request DEL, including an object identifier ID, to storage node 110. The delete request DEL may be a request to delete data corresponding to an object reference with the object identifier ID.

[0119] In operation S302, storage node 110 searches the object hash table O2H for a hash value "H" corresponding to the received object identifier ID. Operation S302 is similar to that described above, therefore, additional description will be omitted to avoid redundancy. In operation S303, storage node 110 sends a deletion request DELa, including the found hash value "H", to the first storage device 120. The deletion request DELa may also include the object identifier ID.

[0120] In operation S304, the first storage device 120 determines whether there is an entry in the hash-physical table H2P corresponding to the received hash value "H". The absence of an entry in the hash-physical table H2P corresponding to the received hash value "H" may mean that the data corresponding to the hash value "H" is not stored in the non-volatile memory 122. In an embodiment, a reference count of "0" for the entry corresponding to the hash value "H" in the hash-physical table H2P can have the same meaning as the absence of an entry corresponding to the hash value "H".

[0121] If no entry corresponding to the received hash value "H" exists in the hash-physical table H2P, in operation S305, the first storage device 120 may return an error to the storage node 110. In operation S306, the storage node 110 may perform error handling. For example, error handling may include resending the delete request DEL or sending an error message to the client.

[0122] If an entry corresponding to the received hash value "H" exists in the hash-physical table H2P, in operation S307, the first storage device 120 decreases the reference count RC of the entry corresponding to the hash value "H" in the hash-physical table H2P by a given value (e.g., "1"). In an embodiment, if this decrease reduces the reference count RC to 0, the first storage device 120 searches for the physical address PA corresponding to the hash value "H" in the hash-physical table H2P and schedules the location associated with the physical address PA in the non-volatile memory 122 to be deleted or marked as invalid.

[0123] In operation S308, the first storage device 120 may send completion information regarding the deletion request DELa to the storage node 110. In operation S309, the storage node 110 may send completion information regarding the deletion request DEL to the client server 101.

[0124] Figure 12 It is used to describe according to Figure 11 The flowchart below illustrates the deletion operation. For the sake of brevity and ease of description, details regarding the deletion operation are omitted. Figure 12 The flowchart deletion operation removes unnecessary components. In the embodiment, components omitted in... Figure 11 The processing performed in operations S304 and S305.

[0125] Reference Figure 1 , Figure 11 and Figure 12 Client server 101 sends a first deletion request DEL1, including a first object identifier ID1, to storage node 110. The first deletion request DEL1 may be a request to delete first data DT1 corresponding to the object reference of the first object identifier ID1.

[0126] The hash module 113 of storage node 110 searches the object-hash table O2H for a first hash value H1 corresponding to the received first object identifier ID1. This has been described above, so further description will be omitted to avoid redundancy. The host interface circuitry 116 of storage node 110 can send a first delete request DEL1a, including the first hash value H1, to the first storage device 120.

[0127] The host interface circuit 121i of the first storage device 120 receives a first deletion request DEL1a including a first hash value H1. The deduplication manager 121d searches for a first reference count RC1 corresponding to the first hash value H1 in the second hash-physical table H2P-2. In response to the first deletion request DEL1a, the deduplication manager 121d updates the second hash-physical table H2P-2 to a third hash-physical table H2P-3 by decreasing the first reference count RC1 corresponding to the first hash value H1. For example, the deduplication manager 121d can decrease the first reference count RC1 by 1.

[0128] In this embodiment, in the hash-physical table H2P-2, a first reference count RC1 of "2" indicates that the number of objects referencing the first data DT1 associated with the first hash value H1 corresponding to the first reference count RC1 is "2". Then, because the first data DT1 referenced by the object with the first object identifier ID1 is deleted, the number of objects referencing the first data DT1 associated with the first hash value H1 changes to "1". Therefore, the value of the first reference count RC1 decreases to "1".

[0129] In this embodiment, storage node 110 can update the object-hash table O2H. For example, as... Figure 12 As shown, the object-hash table O2H can be updated by releasing the mapping of the first object identifier ID1 targeted by the deletion request from the first object identifier ID1 and the second object identifier ID2 corresponding to the first hash value H1. For example, the object-hash table O2H can be updated such that the entry including the first object identifier ID1 is no longer mapped to the first hash value H1, or the entry is deleted.

[0130] Figure 13 It is shown Figure 1 A block diagram illustrating the operation of the storage device. In this embodiment, reference will be made to... Figure 13 The garbage collection operation of the first storage device 120 is described. In an embodiment, this can be understood by referring to... Figure 3 The described flash conversion layer FTL is used to perform Figure 13 The garbage collection operation in the process.

[0131] Reference Figure 1 , Figure 5 and Figure 13 In operation S401, the first storage device 120 selects a source block and a destination block. For example, each of the non-volatile memories 122 may include multiple memory blocks. The storage controller 121 of the first storage device 120 can select the source block and the destination block from the multiple memory blocks. In an embodiment, the destination block is a free memory block.

[0132] In operation S402, the variable "k" is set to "1". The variable "k" is used to describe the iterative operation according to embodiments of this disclosure and is not intended to limit the scope of the invention.

[0133] In operation S403, the first storage device 120 determines whether the k-th reference count RC_k is "0". For example, the k-th reference count RC_k may indicate the reference count of the physical address corresponding to the k-th physical page of the source block. The first storage device 120 may determine whether the k-th reference count RC_k of the hash-physical table H2P is "0".

[0134] A reference count RC_k of "0" can mean that no object is referencing data stored in the physical page associated with the reference count RC_k (i.e., the kth physical page of the source block). When the reference count RC_k is "0", in operation S404, the first storage device 120 can determine that the kth physical page of the source block is an invalid page.

[0135] A non-zero k-th reference count RC_k can mean that an object is referencing data stored in the physical page associated with the k-th reference count RC_k (i.e., the k-th physical page of the source block). In operation S405, when the k-th reference count RC_k is not zero, the first storage device 120 determines that the k-th physical page of the source block is a valid page.

[0136] In operation S406, it is determined whether the variable "k" is the maximum value. For example, it is determined whether the above valid / invalid determination operation should be performed on all pages included in the source block. When the variable "k" is not the maximum value, in operation S407, the variable "k" can be increased by "1", and then the first storage device 120 executes operation S403.

[0137] When the variable "k" is at its maximum value, in operation S408, the first storage device 120 migrates the physical pages determined to be valid from the source block to the destination block. In operation S409, the first storage device 120 can use the source block as a free block. In this embodiment, the migration includes migrating the physical pages determined to be valid from the source block to the destination block and migrating the valid pages from another block to the destination block.

[0138] As described above, the storage device according to embodiments of this disclosure can check the number of objects referencing data based on a reference count. Therefore, deduplication functionality can be provided through the storage device.

[0139] Figure 14 It is shown Figure 1 A flowchart illustrating the operation of the server system. For ease of description, additional descriptions associated with the above components will be omitted to avoid redundancy. See also... Figure 1 and 14 In operation S501, the client server 101 sends a write request PUT to the storage node 110. The write request PUT may include an object identifier ID, a data length len, a data DT, and a flag FG. In this embodiment, the flag FG refers to information used to enable / disable the deduplication function of the first storage device 120.

[0140] Storage node 110 and first storage device 120 can execute operations S502, S503, S504, S505, S506, and S507. Except for using the flag FG, operations S502, S503, S504, S505, S506, and S507 are... Figure 6 Operations S102, S103, S104, S105, S106 and S107 are similar, therefore, additional descriptions will be omitted to avoid redundancy.

[0141] When the result of operation S505 indicates "match", in operation S508, the first storage device 120 determines whether there is an entry in the hash-physical table H2P corresponding to the received hash value "H". When there is no entry in the hash-physical table H2P corresponding to the received hash value "H", the first storage device 120 can execute operations S510, S511, S512, and S513. Operations S510, S511, S512, and S513 are similar to... Figure 6 Operations S109, S110, S111, and S112 are omitted here for brevity.

[0142] When an entry corresponding to the received hash value "H" exists in the hash-physical table H2P, in operation S509, the first storage device 120 determines whether the flag FG is set. For example, the flag FG may refer to information used to disable deduplication of data referencing a specific object. That is, regarding a specific write request, if the flag FG is set, regardless of whether the data DT is stored in the non-volatile memory 122, the data DT corresponding to the specific write request can be stored in the non-volatile memory 122 (i.e., duplicate data can be stored therein). Regarding a specific write request, if the flag FG is not set (e.g., cleared), refer to the above. Figure 6 The given description states that identical data will not be stored; that is, duplicate data will not be stored.

[0143] In other words, without setting the flag FG, the first storage device 120 performs operation S513 without storing the data DT. Conversely, with the flag FG set, the first storage device 120 stores the data DT through operations S510 to S512. In this case, the data DT (i.e., duplicate data) can be stored in the non-volatile memory 122. In embodiments, the flag FG can be set on an I / O basis. The flag FG can be configured to ensure the reliability of data referenced by a specific object.

[0144] Then, the first storage device 120, storage node 110, and client server 101 can execute operations S515, S516, and S517. Operations S515, S516, and S517 are similar to... Figure 6 Operations S113, S114, and S115 will be omitted to avoid redundancy.

[0145] Figure 15 It is used to describe according to Figure 14 The flowchart below illustrates the operation. For the sake of brevity and ease of description, further details are omitted. Figure 14 The flowchart write operation does not require components. In the embodiment, for the sake of brevity of the drawings and for ease of description, the description will be based on... Figure 14 The configuration disables the deduplication function during the operation of the flowchart.

[0146] Reference Figure 1 , Figure 14 and Figure 15 The client server 101 sends a second write request PUT2 to the storage node 110. The second write request PUT2 may include a second object identifier ID2, first data DT1, and flag FG.

[0147] The hash module 113 of storage node 110 calculates a first hash value H1 by performing a hash operation on the first data DT. The host interface circuit 116 of storage node 110 sends a second write request PUT2a, which includes the first hash value H1, to the first storage device 120. The second write request PUT2a may also include a second object identifier ID2, the first data DT1, and a flag FG.

[0148] The host interface circuit 121i of the first storage device 120 can receive a second write request PUT2a including a first hash value H1. The deduplication manager 121d of the first storage device 120 determines whether an entry corresponding to the first hash value H1 exists in the hash-physical table H2P. Figure 15 In one embodiment, an entry corresponding to the first hash value H1 may exist in the hash-physical table H2P. In this case, the deduplication manager 121d allocates a second physical address PA2 in response to the set flag FG. For example, in Figure 8 In this embodiment, for deduplication, if an entry corresponding to a specific hash value exists in the hash-physical table H2P, the write operation for the data is omitted. Conversely, in Figure 15 In the embodiment, because the deduplication function is disabled by setting the flag FG, the deduplication manager 121d performs a write operation on the data even if an entry corresponding to a specific hash value exists in the hash-physical table H2P.

[0149] The memory interface circuit 121j of the first storage device 120 sends the program command CMD_P, the second physical address PA2, and the first data DT1 to at least one of the non-volatile memories 122. At least one of the non-volatile memories 122 can store the first data DT1 in a region corresponding to the second physical address PA2. In this case, the first data DT1 (i.e., duplicate data) can be stored in different regions.

[0150] The deduplication manager 121d can generate a fourth hash-physical table H2P-4 by adding entries corresponding to the first hash value H1 to the hash-physical table H2P-3. In an embodiment, the reference count of the added entries can replace information about the object identifier.

[0151] In this embodiment, as described above, storage node 110 can update object-hash table O2H; therefore, additional descriptions will be omitted to avoid redundancy.

[0152] As described above, according to embodiments of this disclosure, deduplication functionality for data reliability can be provided by disabling deduplication functionality relative to data referenced by a specific object.

[0153] Figure 16 It is shown Figure 1 A flowchart illustrating the operation of the server system. For ease of description, additional descriptions associated with the above components will be omitted to avoid redundancy. See also... Figure 1 and 16 In operation S601, the client server 101 sends a write request PUT to the storage node 110, which includes the object identifier ID, the data length len, and the data DT.

[0154] In this embodiment, the length of the data received from client server 101 and referenced by an object can be varied. In this case, a computationally intensive hashing module could be used to compute a suitable hash value for the various data lengths. Such a hashing module could lead to performance degradation and increased area. Therefore, the data received from client server 101 can be segmented into segments of size with a specific reference value REF before processing.

[0155] For example, in operation S602, storage node 110 determines whether the received data length len is greater than the reference size REF. When the received data length len is not greater than the reference size REF, in operation S603, storage node 110 and the first storage device 120 perform a write operation on the data DT. In this embodiment, the write operation in operation S603 is similar to the reference size REF. Figures 1 to 15The write operation between the storage node 110 and the first storage device 120.

[0156] When the received data length len is greater than the reference size REF, in operation S604, storage node 110 divides the data DT by the reference size REF. For example, the data DT can be divided into two or more data items (e.g., DT1a, DT1b, ..., DT1n), where each data item is less than or equal to the reference size REF. Then, in operations S605a to S605n, storage node 110 and the first storage device 120 perform write operations on the divided data respectively. The write operations in operations S605a to S605n can be similar to the reference size REF. Figures 1 to 15 The described write operation is between storage node 110 and first storage device 120. In embodiments, even when data is partitioned from the same data, the partitioned data may be stored in different storage devices based on the calculated hash value. For example, a first segment of data partitioned from the first data may be stored in either the first storage device 120 or the second storage device 130 based on the calculated hash value.

[0157] Then, in operation S607, storage node 110 updates the object-hash table O2H. For example, storage node 110 can reconfigure the object identifiers of each segment of data, calculate the hash value of each segment of data, and update the object-hash table O2H based on the mapping between the reconfigured object identifiers and the calculated hash values. For example, if data hashed to a first hash value is segmented into first data and second data, a hash operation performed on the first data will result in a second hash value that is different from the first hash value, and a hash operation performed on the second data will result in a third hash value that is different from both the first and second hash values. The first entry in the object-hash table O2H, which includes a specific object identifier and a first hash value, can be replaced by a second entry that includes a specific object identifier and a second hash value, and a third entry that includes a specific object identifier and a third hash value. In operation S608, storage node 110 can send completion information to client server 101.

[0158] As described above, when the size of the data received from the client server 101 is greater than the reference size REF, the client server 101 can split the data into segments with the reference size REF, or split the data based on the reference size REF. In this case, the size and computational cost of the hash module used to calculate the hash value are reduced, thus providing a server system with improved performance.

[0159] Figure 17 It is shown Figure 1 A flowchart illustrating the operation of the server system. For ease of description, additional descriptions associated with the above components will be omitted to avoid redundancy. See also... Figure 1 and 17 In operation S701, the client server 101 sends a read request GET, including the object identifier ID, to the storage node 110.

[0160] In operation S702, storage node 110 determines whether the data corresponding to the received object identifier ID has been segmented. For example, storage node 110 may base its determination on a reference... Figure 16 The described write method segments specific data and can store the segmented data in a storage device.

[0161] When it is determined that the data corresponding to the received object identifier ID has not been segmented, in operation S703, storage node 110 and first storage device 120 perform a read operation. The read operation can be performed in conjunction with a reference... Figures 1 to 15 The read operations described are similar, therefore, additional descriptions will be omitted to avoid redundancy.

[0162] When it is determined that the data corresponding to the received object identifier ID has been segmented, in operation S704, storage node 110 searches for a hash value based on the object identifier reconfigured relative to the object identifier ID. In operations S705a to S705n, storage node 110 and the first storage device 120 perform read operations on the found hash values ​​Ha to Hn, respectively. Figures 1 to 15 The read operations described are similar, so additional descriptions will be omitted to avoid redundancy.

[0163] In operation S706, storage node 110 merges multiple data received through read operations. For example, read fragments of split data can be combined to generate merged data. In operation S707, storage node 110 can send the merged data as data DT corresponding to object identifier ID to client server 101.

[0164] Figure 18 and 19 It shows a reference. Figure 16 and 17 A diagram illustrating an example of data segmentation. (See reference...) Figure 18 and 19 The data corresponding to the first object identifier ID1 can be 10KB in size. With a reference size REF of 4KB, the data corresponding to the first object identifier ID1 is divided into 4KB units, and object identifiers ID1a, ID1b, and ID1c are assigned to the divided data (or segments). For example, when the data size is not divisible by the reference size REF, the data can be divided into complete 4KB units and a remaining portion smaller than 4KB.

[0165] In this embodiment, the method of data segmentation is determined such that the size of the segmented data is smaller than a reference size. For example, such as Figure 18 As shown, the data corresponding to the "ID1a" segment can be 4KB in size, the data corresponding to the "ID1b" segment can be 4KB in size, and the data corresponding to the "ID1c" segment can be 2KB in size.

[0166] Conversely, the method of data partitioning is determined such that the size of the partitioned data is equal to the reference size. For example, such as Figure 19 As shown, the data segments corresponding to "ID1a", "ID1b", and "ID1c'" can each have 4KB. In this case, a portion (i.e., 2KB) of the data segment corresponding to "ID1c'" can be actual data, while the remaining portion (i.e., 2KB) can be pseudo-data or padding data. For example, the size of the data corresponding to the first object identifier ID1 can be increased by appending pseudo-data or padding data so that the result is divisible by the reference size REF.

[0167] As described above, in a communication environment where data size is variable, to reduce the burden of hash operations, the storage node 110 according to this disclosure can segment and manage data based on a reference size REF.

[0168] Figure 20 This is a diagram illustrating a server system according to an embodiment of the present disclosure. Figure 21 It is used to describe in Figure 20 A diagram illustrating the hash range managed at each storage node of the server system. (Refer to...) Figure 20 The server system 200 may include a client server 201 and multiple storage servers 202, 203 and 204. The client server 201 and the multiple storage servers 202, 203 and 204 can communicate with each other via a network NT.

[0169] The multiple storage servers 202, 203, and 204 may include multiple storage nodes 211, 212, and 213 and multiple storage devices 221, 222, 223, 224, 225, and 226. The components of the multiple storage servers 202, 203, and 204 (e.g., the multiple storage nodes 211, 212, and 213 and the multiple storage devices 221, 222, 223, 224, 225, and 226) are similar to those described above, therefore, additional descriptions will be omitted to avoid redundancy.

[0170] In the embodiments, Figure 20In this example, the number of client servers, storage servers, storage nodes, and storage devices are just examples, and these numbers can be varied (e.g., increased / decreased).

[0171] Storage nodes 211, 212, and 213 can be configured to manage data corresponding to different object identifiers. For example, client server 201 can access data corresponding to a first object identifier through first storage node 211, data corresponding to a second object identifier through second storage node 212, and data corresponding to a third object identifier through third storage node 213.

[0172] The first storage node 211 included in the first storage server 202 can be configured to manage storage devices 221 and 222. The second storage node 212 included in the second storage server 203 can be configured to manage storage devices 223 and 224. The third storage node 213 included in the third storage server 204 can be configured to manage storage devices 225 and 226.

[0173] In an embodiment, a first storage node 211 may be configured to access storage devices 223, 224, 225, and 226 managed by a second storage node 212 and a third storage node 213, which are different from the first storage node 211. For example, the first storage node 211 may access the storage devices 223, 224, 225, and 226 managed by the second storage node 212 and the third storage node 213 via a network NT or any other separate communication interface. Similarly, the second storage node 212 may access the storage devices 221, 222, 225, and 226 managed by the first storage node 211 and the third storage node 213 via a network NT or any other separate communication interface. Similarly, the third storage node 213 may access the storage devices 221, 222, 223, and 224 managed by the first storage node 211 and the second storage node 212 via a network NT or any other separate communication interface.

[0174] In this embodiment, the first to third storage nodes 211, 212, and 213 are configured to manage the hash range of storage devices 221 to 226. For example, as Figure 21As shown, the first storage node 211 manages hash ranges HR_11 and HR_12 corresponding to storage devices 221 and 222, respectively; the second storage node 212 manages hash ranges HR_21 and HR_22 corresponding to storage devices 223 and 224, respectively; and the third storage node 213 manages hash ranges HR_31 and HR_32 corresponding to storage devices 225 and 226, respectively. In this embodiment, the hash ranges HR_11, HR_12, HR_21, HR_22, HR_31, and HR_32 of storage devices 221 to 226 are different. For example, in this embodiment, the hash ranges HR_11, HR_12, HR_21, HR_22, HR_31, and HR_32 do not overlap with each other.

[0175] According to embodiments of this disclosure, server system 200 provides deduplication functionality to multiple objects that reference the same data. For example, as described above, assume that client server 201 accesses data associated with a first object identifier through a first storage node 211, data associated with a second object identifier through a second storage node 212, and data associated with a third object identifier through a third storage node 213.

[0176] In this embodiment, during a write operation, the client server 201 sends a write request including a first object identifier ID1 and first data DT1 to the first storage node 211. The first storage node 211 calculates a first hash value H1 by performing a hash operation on the first data DT. The first storage node 211 determines the hash range to which the first hash value H1 belongs. The first storage node 211 may send the write request including the first data DT1 and the first hash value H1 to a storage device that has the hash range to which the first hash value H1 belongs. For example, if the first hash value H1 is included in the hash range HR_21 of the storage device 223 managed by the second storage node 212, the first storage node 211 may send the write request including the first data DT1 and the first hash value H1 to the storage device 223. The first storage node 211 may update the object-hash table O2H based on the first object identifier ID1 and the first hash value H1.

[0177] In this embodiment, during a read operation, the client server 201 sends a read request including a first object identifier ID1 to the first storage node 211. The first storage node 211 searches for a first hash value H1 corresponding to the first object identifier ID1 in the object-hash table O2H. As described above with reference to the write operation, the first storage node 211 can determine the hash range to which the first hash value H1 belongs, and can send the read request including the first hash value H1 to a storage device having the hash range to which the first hash value H1 belongs.

[0178] In other words, the client server 201 can classify and manage multiple storage nodes 211 to 213 based on object identifiers, and the multiple storage nodes 211 to 213 can manage the storage device storing the data based on the hash value of the data. As described above, the storage device can provide deduplication functionality based on the hash value and reference count.

[0179] Figure 22 It is shown Figure 20 A flowchart illustrating the operation of the server system. For ease of description, additional descriptions associated with the above components will be omitted to avoid redundancy. See also... Figure 20 and 22 In operation S801, client server 201 sends a write request PUT including a first object identifier ID, data length len, and first data DT to first storage node 211. For example, client server 201 can select the storage server or storage node to be accessed based on the object identifier. For example, a hash value can be determined from the object identifier, and the storage node associated with the hash range including the hash value can be selected as the storage node.

[0180] In operation S802, the first storage node 211 performs a hash operation on the first data DT1 to calculate a first hash value H1. In operation S803, the first storage node 211 determines whether the first hash value H1 is included in the hash range managed by the first storage node 211 itself. For example, as referred to... Figure 21 As described, the hash range managed by the first storage node 211 is "HR_11" and "HR_12". The first storage node 211 can determine whether the first hash value H1 is included in the hash ranges HR_11 and HR_12.

[0181] When the first hash value H1 is included in the hash range managed by the first storage node 211, in operation S804, the first storage node 211 and the storage device 221 or 222 perform a write operation.

[0182] When the first hash value H1 is not included in the hash ranges HR_11 and HR_12 managed by the first storage node 211, in operation S805, the first storage node 211 determines the hash range to which the first hash value H1 belongs. For example, if the first hash value H1 is included in hash range HR_21 or hash range HR_22, the first storage node 211 determines that the first hash value H1 corresponds to the second storage node 212. If the first hash value H1 is included in hash range HR_31 or hash range HR_32, the first storage node 211 may determine that the first hash value H1 corresponds to the third storage node 213.

[0183] When it is determined that the first hash value H1 is included in the hash range HR21 or HR22 managed by the second storage node 212, in operation S806, the first storage node 211 sends a write request PUT to the second storage node 212. In operation S807, the second storage node 212 calculates the first hash value H1. In operation S808, the second storage node 212 and the storage device 223 or 224 perform a write operation.

[0184] When it is determined that the first hash value H1 is included in the hash range HR31 or HR32 managed by the third storage node 213, in operation S809, the first storage node 211 sends a write request PUT to the third storage node 213. In operation S810, the third storage node 213 calculates the first hash value H1. In operation S811, the third storage node 213 and the storage device 225 or 226 perform a write operation.

[0185] In the embodiment, the write operation in operation S804, the write operation in operation S808, and the write operation in operation S811 are similar to those described in the reference. Figures 1 to 15 The description of the write operation (i.e., the write operation that provides deduplication functionality using hash values ​​and reference counts) will therefore omit any additional description to avoid redundancy.

[0186] After completing the write operation in operation S808, in operation S812, the second storage node 212 can send completion information to the first storage node 211. After completing the write operation in operation S811, in operation S813, the third storage node 213 can send completion information to the first storage node 211.

[0187] When the write operation in operation S804 has been completed or the completion information sent in operation S812 or operation S813 has been received, in operation S814, the first storage node 211 can send the completion information to the client server 201.

[0188] In operation S815, the first storage node 211 updates the object-hash table O2H. For example, the first storage node 211 may update the object-hash table O2H based on the mapping information between the first object identifier ID1 and the first hash value H1. In an embodiment, then, if the first storage node 211 receives a read request for the first object identifier ID1, the first storage node 211 may search for the first hash value H1 in the updated object-hash table O2H and may perform a read operation based on the found first hash value H1.

[0189] Figure 23 It is shown Figure 20A flowchart illustrating the operation of the server system. For ease of description, additional descriptions associated with the above components will be omitted to avoid redundancy. See also... Figure 20 and 23 In operation S901, the client server 201 sends a read request GET including the first object identifier ID1 to the first storage node 211.

[0190] In operation S902, the first storage node 211 searches for the first hash value H1 corresponding to the first object identifier ID1 in the object hash table O2H.

[0191] In operation S903, the first storage node 211 determines whether the first hash value H1 is included in the hash range managed by the first storage node 211 itself. Operation S903 and... Figure 22 The operation is similar to S803, so additional descriptions will be omitted to avoid redundancy.

[0192] When the first hash value H1 is included in the hash range managed by the first storage node 211, in operation S904, the first storage node 211 and the first storage device 221 or 222 perform a read operation.

[0193] When the first hash value H1 is not included in the hash ranges HR_11 and HR_12 managed by the first storage node 211, in operation S905, the first storage node 211 determines the hash range to which the first hash value H1 belongs. Operation S905 is similar to... Figure 22 The operation S805 is used, therefore, additional descriptions will be omitted to avoid redundancy.

[0194] When it is determined that the first hash value H1 is included in the hash range HR21 or HR22 of the second storage node 212, in operation S906, the first storage node 211 sends a read request GETa including the first hash value H1 to the second storage node 212. In operation S907, the second storage node 212 and the storage device 223 or 224 perform a read operation.

[0195] When it is determined that the first hash value H1 is included in the hash range HR31 or HR32 of the third storage node 213, in operation S908, the first storage node 211 sends a read request GETa including the first hash value H1 to the third storage node 213. In operation S909, the third storage node 213 and the storage device 225 or 226 perform a read operation.

[0196] In the embodiment, the read operation in operation S904, the read operation in operation S907, and the read operation in operation S909 are similar to those described in the reference. Figures 1 to 15The description of the read operation (i.e., the read operation using the hash value) will be omitted to avoid redundancy.

[0197] After completing the read operation in operation S907, in operation S910, the second storage node 212 sends the read data (i.e., the first data DT1) to the first storage node 211. After completing the read operation in operation S909, in operation S911, the third storage device 213 sends the read data (i.e., the first data DT1) to the first storage node 211.

[0198] When the read operation in operation S904 is completed or the first data read in operation S910 or operation S911 is received, in operation S912, the first storage node 211 sends the first data DT1 to the client server 201.

[0199] While an embodiment of client server 201 performing an access (e.g., write / read operation) associated with the first object identifier ID1 via first storage node 211 has been described, the scope of the invention is not limited thereto. For example, in the case where client server 201 performs an access associated with the second object identifier ID2 via second storage node 212, client server 201 may send a request corresponding to the second object identifier ID2 to second storage node 212. Second storage node 212 may select the storage node or storage device to be accessed based on a second hash value (i.e., for write operations, the second hash value is the hash value of the second data; for read operations, the second hash value is the hash value associated with the second object identifier ID2).

[0200] Reference Figure 20 and Figure 22 An embodiment of a specific storage node accessing a storage device belonging to another storage node has been described, but this disclosure is not limited thereto. For example, a specific storage node may access a storage device belonging to another storage node without going through the other storage node. Alternatively, a specific storage node may access a storage device belonging to another storage node through a component (e.g., a switch) of another storage node.

[0201] As described above, according to at least one embodiment of this disclosure, a storage device and a storage server are provided capable of implementing deduplication based on the hash value of data. In this case, the amount of information managed by the storage node can be reduced, and the performance of the storage node can be improved.

[0202] Figure 24This is a block diagram illustrating a data center employing a storage device according to an embodiment of the present disclosure. The data center 2000, as a facility that maintains various types of data and provides various services related to that data, may be referred to as a "data storage center." The data center 2000 may be a system for search engines or database management, or it may be a computing system used in various organizations. The data center 2000 may include multiple application servers 2100_1 to 2100_n and multiple storage servers 2200_1 to 2200_m. The number of application servers 2100_1 to 2100_n and the number of storage servers 2200_1 to 2200_m may be varied or modified.

[0203] For ease of description, an example of the first storage server 2200_1 will be described below. Each of the remaining storage servers 2200_2 to 2200_m and the plurality of application servers 2100_1 to 2100_n may have a structure similar to that of the first storage server 2200_1.

[0204] The first storage server 2200_1 may include a processor 2210_1, a memory 2220_1, a switch 2230_1, a network interface connector (NIC) 2240_1, and a storage device 2250_1. The processor 2210_1 can perform the overall operation of the first storage server 2200_1. The memory 2220_1 can store various instructions or data under the control of the processor 2210_1. The processor 2210_1 can be configured to access the memory 2220_1 to execute various instructions or process data. In embodiments, the memory 2220_1 may include at least one of various memory devices such as DDR SDRAM (Dual Data Rate Synchronous DRAM), HBM (High Bandwidth Memory), HMC (Hybrid Memory Cube), DIMM (Dual In-line Memory Module), Optane DIMM, and NVDIMM (Non-Volatile DIMM)).

[0205] In this embodiment, the number of processors 2210_1 and the number of memories 2220_1 included in the first storage server 2200_1 can be varied or modified. In this embodiment, the processors 2210_1 and memories 2220_1 included in the first storage server 2200_1 can form processor-memory pairs, and the number of processor-memory pairs included in the first storage server 2200_1 can be varied or modified. In this embodiment, the number of processors 2210_1 and the number of memories 2220_1 included in the first storage server 2200_1 can be different. The processor 2210_1 may include a single-core processor or a multi-core processor.

[0206] Under the control of processor 2210_1, switch 2230_1 can selectively connect processor 2210_1 to storage device 2250_1, or selectively connect NIC 2240_1 to storage device 2250_1.

[0207] NIC 2240_1 can connect the first storage server 2200_1 to the network NT. NIC 2240_1 may include a network interface card, network adapter, etc. NIC 2240_1 can connect to the network NT via a wired interface, wireless interface, Bluetooth interface, or optical interface. NIC 2240_1 may include internal memory, DSP, host bus interface, etc., and can connect to processor 2210_1 or switch 2230_1 via the host bus interface. The host bus interface may include at least one of various interfaces, such as ATA (Advanced Technology Attachment) interface, SATA (Serial ATA) interface, e-SATA (External SATA) interface, SCSI (Small Computer Small Interface), SAS (Serial Attached SCSI) interface, PCI (Peripheral Component Interconnect) interface, PCIe (PCI Fast) interface, NVMe (NVM Fast) interface, IEEE 1394 interface, USB (Universal Serial Bus) interface, SD (Secure Digital) card interface, MMC (Multimedia Card) interface, eMMC (Embedded Multimedia Card) interface, UFS (Universal Flash Memory) interface, eUFS (Embedded Universal Flash Memory) interface, and CF (Compact Flash Memory) card interface. In an embodiment, NIC 2240_1 may be integrated with at least one of processor 2210_1, switch 2230_1, and storage device 2250_1.

[0208] Under the control of processor 2210_1, storage device 2250_1 can store data or output stored data. Storage device 2250_1 may include controller 2251_1, non-volatile memory 2252_1, DRAM 2253_1, and interface 2254_1. In embodiments, storage device 2250_1 may also include a security element (SE) for security or privacy.

[0209] Controller 2251_1 can control the overall operation of storage device 2250_1. In an embodiment, controller 2251_1 may be SRAM. In response to a signal received through interface 2254_1, controller 2251_1 may store data in non-volatile memory 2252_1 or may output data stored in non-volatile memory 2252_1. In an embodiment, controller 2251_1 may be configured to control non-volatile memory 2252_1 based on a switching interface or ONFI.

[0210] DRAM 2253_1 can be configured to temporarily store data to be stored in or read from non-volatile memory 2252_1. DRAM 2253_1 can be configured to store various data (e.g., metadata and mapping data) required for the operation of memory controller 2251_1. Interface 2254_1 can provide physical connectivity between controller 2251_1 and processor 2210_1, switch 2230_1, or NIC 2240_1. In an embodiment, the interface can be implemented to support DAS (Direct Attached Storage) mode, allowing direct connection of storage device 2250_1 via a dedicated cable. In an embodiment, interface 2254_1 can be implemented via a host interface bus based on at least one of the various interfaces described above.

[0211] The above components of the first storage server 2200_1 are provided as examples, and this disclosure is not limited thereto. The above components of the first storage server 2200_1 can be applied to each of the remaining storage servers 2200_2 to 2200_m and each of the other application servers 2100_1 to 2100_n. In embodiments, storage devices 2150_1 to 2150_n of the application servers 2100_1 to 2100_n can be optionally omitted.

[0212] Multiple application servers 2100_1 to 2100_n and multiple storage servers 2200_1 to 2200_m can communicate with each other via a network NT. The network NT can be implemented using Fibre Channel (FC), Ethernet, etc. In this case, FC can be used as the medium for high-speed data transmission, and optical switches providing high performance / high availability can be used. Depending on the access method of the network NT, the storage servers 2200_1 to 2200_m can be configured as file storage, block storage, or object storage.

[0213] In this embodiment, the network NT can be a storage-specific network such as a Storage Area Network (SAN). For example, the SAN can be an FC-SAN implemented using an FC network and according to the FC protocol (FCP). Alternatively, the SAN can be an IP-SAN implemented using a TCP / IP network and according to the iSCSI (or SCSI over TCP / IP or Internet SCSI) protocol. In this embodiment, the network NT can be a general-purpose network such as a TCP / IP network. For example, the network NT can be implemented according to protocols such as FCoE (FC over Ethernet), NAS (Network Attached Storage), or NVMe-oF (NVMe over Fibre).

[0214] In an embodiment, at least one of the plurality of application servers 2100_1 to 2100_n may be configured to access at least one of the remaining application servers 2200_1 to 2200_m via network NT.

[0215] For example, the first application server 2100_1 can store user or client-requested data on at least one of multiple storage servers 2200_1 to 2200_m via a network NT. Alternatively, the first application server 2100_1 can obtain user or client-requested data from at least one of the multiple storage servers 2200_1 to 2200_m via a network NT. In this case, the first application server 2100_1 can be implemented using a network server, a database management system (DBMS), etc.

[0216] In other words, the processor 2110_1 of the first application server 2100_1 can access the memory (e.g., 2120_n) or storage device (e.g., 2150_n) of another application server (e.g., 2100_n) via the network NT. Alternatively, the processor 2110_1 of the first application server 2100_1 can access the memory 2220_1 or storage device 2250_1 of the first storage server 2200_1 via the network NT. In this way, the first application server 2100_1 can perform various operations on data stored in the remaining application servers 2100_2 to 2100_n or multiple storage servers 2200_1 to 2200_m. For example, the first application server 2100_1 can execute or issue instructions for moving or copying data between the remaining application servers 2100_2 to 2100_n or between multiple storage servers 2200_1 to 2200_m. In this scenario, data destined for movement or copying can be moved from storage devices 2250_1 to 2250_m of storage servers 2200_1 to 2200_m to storage devices 2120_1 to 2120_n of application servers 2100_1 to 2100_n, either via storage devices 2220_1 to 2220_m or directly. Data transmitted over the network NT can be encrypted for security or privacy purposes.

[0217] In this embodiment, storage servers 2200_1 to 2200_m may be referenced. Figures 1 to 24 The storage server described, and the storage devices 2250_1 to 2250_m included in the storage servers 2200_1 to 2200_m, can be referred to as Figures 1 to 24The storage devices described. That is, the storage servers 2200_1 to 2200_m or storage devices 2250_1 to 2250_m of the data center 2000 shown in Figure 25 can be configured to support the reference. Figures 1 to 23 The description includes write, read, and delete operations.

[0218] According to this disclosure, a storage device with improved performance and reduced cost that supports deduplication is provided, a method of operating the storage device, and a method of operating the storage server.

[0219] Although this disclosure has been described with reference to embodiments thereof, those skilled in the art should understand that various changes and modifications can be made to this disclosure without departing from the spirit and scope thereof.

Claims

1. A method of operating a storage device, the method comprising: Receive a first write request from an external device, including an object identifier and data; Perform a hash operation on the data to generate a first hash value; Determine whether the first entry in the table associated with the first hash value is empty; When it is determined that the first entry is empty, the data is stored in the area of ​​the storage device corresponding to the first physical address, and the first entry is updated to include the first physical address and the reference count; as well as When it is determined that the first entry is not empty, the reference count included in the first entry is increased, and the storage operation associated with the data is not performed. When no entry associated with the hash value exists in the table, an error message is returned to the external device; and The storage device described therein does not perform a mapping operation between logical addresses and physical addresses.

2. The method according to claim 1, wherein, The first write request also includes a second hash value of the data, and the method further includes: When the first hash value does not match the second hash value, an error message is sent to the external device.

3. The method according to claim 1, further comprising: Receive a read request from the external device, including the object identifier and the first hash value; The data is read based on the first physical address of the first entry corresponding to the first hash value; Perform a hash operation on the data to generate a second hash value; as well as When the first hash value and the second hash value match, the data is sent to the external device.

4. The method according to claim 3, further comprising: Upon receiving the read request, if it is determined that the table does not include a first entry corresponding to the first hash value, an error message is sent to the external device.

5. The method according to claim 1, further comprising: Receive a deletion request from the external device, including the object identifier and the first hash value; as well as Decrease the reference count of the first entry corresponding to the first hash value.

6. The method according to claim 5, further comprising: When the reference count of the first entry decreases to 0, during the garbage collection operation, the data stored in the region corresponding to the first physical address of the first entry is selected as invalid data.

7. The method according to claim 1, wherein, The reference count represents the number of objects that reference the data.

8. The method according to claim 1, further comprising: Receive a second write request from the external device, including the object identifier, the data, and a flag; Perform the hash operation on the data of the second write request to generate a second hash value that is the same as the first hash value; as well as When the flag is set, the data in the second write request is stored in the area corresponding to the second physical address, and a second entry including the second hash value and the second physical address is added to the table.

9. A method of operating a storage server, the storage server comprising a plurality of storage devices and a storage node configured to manage the plurality of storage devices, the method comprising: At the storage node, a first write request including a first object identifier and first data is received from an external client server. A hash operation is performed on the first data at the storage node to generate a first hash value; At the storage node, a first hash range including the first hash value is determined from multiple hash ranges respectively allocated to the plurality of storage devices; At the storage node, a second write request, including the first object identifier, the first data, and the first hash value, is sent to the first storage device among the plurality of storage devices that corresponds to the first hash range. The hash operation is performed on the first data included in the second write request at the first storage device to generate a second hash value; When the first hash value matches the second hash value and the first entry is empty in the first table, the first data is stored in the area corresponding to the first physical address at the storage device, and the first entry is updated to include the first physical address and the first reference count; When the first hash value matches the second hash value and a first entry corresponding to the first hash value exists in the first table, the first reference count of the first entry is increased at the first storage device, and the first data is not stored. as well as The response is sent from the first storage device to the storage node. The storage device described therein does not perform a mapping operation between logical addresses and physical addresses.

10. The operating method according to claim 9, further comprising: The second table is updated at the storage node based on the first object identifier and the first hash value.

11. The operating method according to claim 10, wherein, Each entry in the first table includes information about the hash value of the data, the physical address of the data, and the reference count of the data, and each entry in the second table includes information about multiple object identifiers and multiple hash values ​​corresponding to the multiple object identifiers respectively.

12. The method of claim 11, further comprising: At the storage node, a first read request including the first object identifier is received from the external client server. At the storage node, the first hash value corresponding to the first object identifier is detected based on the second table; At the storage node, a second read request, including the first object identifier and the first hash value, is sent to the first storage device; The first data is read at the first storage device based on the first physical address of the first entry corresponding to the first hash value; The first data is sent from the first storage device to the storage node; as well as The first data is sent to the external client server at the storage node.

13. The method of claim 9, further comprising: When the first hash value and the second hash value do not match, an error message is sent from the first storage device to the storage node.

14. The method of claim 13, further comprising: The hash operation is performed again on the first data at the storage node to regenerate the first hash value; as well as A third write request, including the regenerated first hash value, is sent to the first storage device.

15. The method of claim 9, further comprising: At the storage node, device size information is received from the plurality of storage devices respectively; as well as Based on the device size information from the plurality of storage devices, the plurality of hash ranges are respectively allocated to the plurality of storage devices at the storage node, wherein the plurality of hash ranges do not overlap with each other.

16. The method of claim 15, further comprising: In response to the first hash value not being included in the plurality of hash ranges, an error message is returned to the external device at the storage node.

17. The method of claim 9, further comprising: When the size of the first data received from the external client server is greater than the reference size, the first data is divided into units of the reference size at the storage node.

18. A storage device, comprising: Non-volatile memory; as well as A storage controller is configured to control the non-volatile memory. The storage controller includes: The memory is configured to store the first table; A hash module configured to perform a hash operation on first data received from an external device to generate a first hash value; and A deduplication manager is configured to search a first table for a first entry corresponding to a first hash value, and to selectively store the first data in the non-volatile memory based on the search results. When the first entry exists in the first table, the deduplication manager increments a first reference count of the first entry without storing the first data in the non-volatile memory. When the first entry does not exist in the first table, the deduplication manager is configured to store the first data in a region corresponding to a first physical address in the non-volatile memory and add the first entry, including the first hash value, the first physical address, and the first reference count, to the first table.

19. The storage device according to claim 18, wherein, The storage controller also includes: A memory interface circuit is configured to communicate with the non-volatile memory, wherein, when the first entry exists in the first table, the first data is not sent to the non-volatile memory via the memory interface circuit.

20. The storage device according to claim 18, wherein, The storage controller also includes: A flash conversion layer is configured to perform a garbage collection operation on the non-volatile memory, wherein, during the garbage collection operation, the flash conversion layer determines data stored at a physical address corresponding to a reference count with a value of 0 among a plurality of reference counts included in the first table as invalid data.

Images