Storage device, operating method of storage device, and operating method of storage system including

Abstract

Disclosed are a memory device, an operating method of the memory device, and an operating method of a memory system including the memory device. The operation method of the storage device connected to a computing fast link (CXL) switch includes: receiving, by the CXL switch, a first access request from a first host; executing an operation corresponding to the first access request; detecting a change in first metadata managed by the first host based on the first access request; and transmitting, through the CXL switch, first update information corresponding to a change in the first metadata to a second host sharing the storage device.

Description

Technical Field

[0001] Embodiments of the present invention described herein relate to a semiconductor memory, and more particularly, to a memory device, an operating method of the memory device, and an operating method of a memory system including the memory device. Background Art

[0002] Semiconductor memory can be classified into: volatile memory, which loses data when power is cut off, such as static random access memory (SRAM) and dynamic random access memory (DRAM); or, non-volatile memory, which retains the data stored therein even when power is cut off, such as flash memory, phase change RAM (PRAM), magnetic RAM (MRAM), resistive RAM (RRAM) and ferroelectric RAM (FRAM). Flash memory is widely used as a large-capacity storage medium. Nowadays, multiple storage devices and multiple hosts are connected in a data center, multiple hosts can share multiple storage devices, and multiple hosts manage multiple storage devices independently of each other. However, this can lead to inconsistencies in metadata managed by different hosts. In order to cope with this situation, multiple hosts maintain consistency of metadata between multiple hosts through separate communications, but separate communications between multiple hosts (i.e., inter-host communications) may cause overhead, ultimately reducing the performance of the entire storage system. Summary of the invention

[0003] Embodiments of the present invention provide a storage device with improved reliability and improved performance, an operating method of the storage device, and an operating method of a storage system including the storage device.

[0004] According to an embodiment, an operating method of a storage device connected to a compute express link (CXL) switch includes: receiving a first access request from a first host through the CXL switch; performing an operation corresponding to the first access request; detecting a change in first metadata managed by the first host based on the first access request; and sending first update information corresponding to the change in the first metadata to a second host sharing the storage device through the CXL switch.

[0005] According to an embodiment, a storage device includes a nonvolatile memory device, a dynamic random access memory (DRAM) device, and a controller, the controller controls the nonvolatile memory device and the DRAM device in response to a first access request received from a first host among a plurality of hosts. The controller includes: a compute express link (CXL) interface circuit, the CXL interface circuit is connected to a CXL switch and receives the first access request from the first host through the CXL switch; a NAND interface circuit, the NAND interface circuit controls the nonvolatile memory device based on the first access request; a DRAM controller, the DRAM controller controls the DRAM device based on the first access request; and a consistency circuit, the consistency circuit detects a change in first metadata managed by the first host in response to the first access request, and sends first update information corresponding to the change in the first metadata to other hosts among the plurality of hosts sharing the storage device through the CXL switch.

[0006] According to an embodiment, an operation method of a storage system includes a compute express link (CXL) switch, a first storage device connected to the CXL switch, a first host connected to the CXL switch, and a second host connected to the CXL switch, the operation method including: sending, by the first host, a first access request corresponding to a storage area of ​​the first storage device to the first storage device through the CXL switch and updating first metadata; receiving, by the first storage device, the first access request through the CXL switch and performing an operation corresponding to the first access request; detecting, by the first storage device, a change in the first metadata and sending, by the CXL switch, first update information corresponding to the change in the first metadata to the second host; and receiving, by the second host, the first update information through the CXL switch and updating second metadata based on the first update information. The first host and the second host share the first storage device, the first host manages a first storage area of ​​the first storage device based on the first metadata, and the second host manages the first storage area of ​​the first storage device based on the second metadata.

[0007] According to an embodiment, a coherence device includes: a form factor configured to be attached to a CXL switch of a storage system; and an interface circuit configured to communicate with the CXL switch through the interface circuit; and a coherence circuit. The coherence circuit is configured to detect a change in first metadata in response to a first memory access request from a first host through the CXL switch; and send first update information corresponding to the change in the first metadata to a second host, the first host and the second host sharing a plurality of storage devices through the CXL switch. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other objects and features of the present invention will become apparent by describing in detail the embodiments of the present invention with reference to the accompanying drawings.

[0009] Figure 1 is a block diagram illustrating a storage system.

[0010] Figure 2 is a block diagram illustrating a storage system according to an embodiment of the present invention.

[0011] Figure 3A and Figure 3B Is used to describe Figure 2 Diagram of the operation of a storage system.

[0012] Figure 4 Is used to describe Figure 2 A diagram of storage areas recognized by multiple hosts.

[0013] Figure 5 It's a picture. Figure 2 A flow chart of the operation of the first storage device.

[0014] Figure 6 It is a diagram of Figure 2 A diagram of a shared table used by a multicast circuit of a first storage device.

[0015] Fig. 7A and Figure 7B Is used to describe Figure 2 Diagram of the operation of a storage system.

[0016] Figure 8 It is used to describe how to update Figure 2 A diagram of the metadata of the second host.

[0017] Fig. 9 The reference is shown in the figure Fig. 7A and Figure 7B A flow chart describing the operation of a storage system.

[0018] FIG. 10A to FIG. 10C Is used to describe Figure 2 Diagram of the operation of a storage system.

[0019] Fig.11 The reference is shown in the figure FIG. 10A to FIG. 10C A flow chart describing the operation of a storage system.

[0020] Fig.12 It's a picture. Figure 2 A diagram of a shared table included in a first storage device.

[0021] Fig.13 Is used to describe Figure 2 Diagram of the operation of a storage system.

[0022] Fig.14 It is a diagram showing the Fig.13 A flow chart of the operation of a storage system of an embodiment.

[0023] FIG. 15A to FIG. 15D Is used to describe Figure 2 Diagram of the operation of a storage system.

[0024] Fig.16 It's a picture. FIG. 15A to FIG. 15D A flow chart of the operation of a storage system.

[0025] Fig.17 is a block diagram illustrating a storage system according to an embodiment of the present invention.

[0026] Fig.18 It's a picture. Fig.17 A flow chart of the operation of a storage system.

[0027] Fig.19 It's a picture. Fig.17 A flow chart of the operation of a storage system.

[0028] Fig. 20 is a block diagram illustrating the topology of a storage system according to an embodiment of the present invention.

[0029] Fig.21 is a block diagram illustrating a data center to which a storage system according to an embodiment of the present invention is applied. DETAILED DESCRIPTION

[0030] Hereinafter, embodiments of the present invention will be described in detail and clearly so that a person skilled in the art can easily practice the present invention.

[0031] Ordinal numbers such as "first", "second", "third", etc. may be simply used as labels for certain elements, steps, etc. to distinguish such elements, steps, etc. from each other. Terms that are not described using "first", "second", etc. in the specification may still be referred to as "first" or "second" in the claims. In addition, a term referenced with a specific ordinal number (e.g., "first" in a specific claim) may be described elsewhere with a different ordinal number (e.g., "second" in the specification or another claim).

[0032] Figure 1 is a block diagram illustrating a storage system. Figure 1 , the storage system 10 may include hosts 11 and 12, a network 13, data servers 14 and 16, and a plurality of storage devices 15 and 17. The hosts 11 and 12 and the data servers 14 and 16 may be connected to the network 13. The data servers 14 and 16 may be connected to the storage devices 15 and 17, respectively. The data servers 14 and 16 may control the storage devices 15 and 17, respectively.

[0033] In an embodiment, the storage system 10 may be a disaggregated storage system. A disaggregated storage system (or architecture) may be a type of data storage in a computer data center. In a disaggregated storage system, storage resources are aggregated separately from computing resources in a centralized storage system (i.e., a computer server) without modifying the internal physical connections. For example, in a disaggregated storage system, computing resources (e.g., a host) and storage resources (e.g., a storage device) may be connected to each other via a network, and one or more of the computing resources and the storage resources may be replaced or exchanged with new or different resources, or added to increase the total capacity, or removed.

[0034] In an embodiment, each of the hosts 11 and 12 can access the storage devices 15 and 17 through the network 13 and the data servers 14 and 16. For example, the first host 11 can access the first storage device 15 through the network 13 and the first data server 14, and can access the second storage device 17 through the network 13 and the second data server 16. Likewise, the second host 12 can access the first storage device 15 through the network 13 and the first data server 14, and can access the second storage device 17 through the network 13 and the second data server 16.

[0035] The hosts 11 and 12 may share the storage space of the storage devices 15 and 17 with each other. For example, each of the hosts 11 and 12 may individually access the storage space of the storage devices 15 and 17. In this case, the first host 11 may identify or manage the storage space of the storage devices 15 and 17 based on the first metadata MD1, and the second host 12 may identify or manage the storage space of the storage devices 15 and 17 based on the second metadata MD2.

[0036] In this case, since the storage devices 15 and 17 are accessed separately by the hosts 11 and 12, the first metadata MD1 and the second metadata MD2 respectively managed by the first host 11 and the second host 12 may be different from each other. However, since the first metadata MD1 and the second metadata MD2 are information about the same storage space (i.e., information about the storage devices 15 and 17), it is preferable to maintain consistency between the first metadata MD1 and the second metadata MD2. Therefore, the hosts 11 and 12 may perform communication for maintaining consistency of the first metadata MD1 and the second metadata MD2 through a separate interface or network 13. When the storage system 10 or the hosts 11 and 12 are operating, communication operations for consistency of metadata may be overhead.

[0037] Figure 2 1 is a block diagram illustrating a storage system according to an embodiment of the present invention. For the sake of simplicity of the drawings and for ease of description, the storage system 100 is depicted as follows: Figure 2 As shown, four hosts H1 to H4 and three storage devices 110 to 130 are included, but the number of hosts or storage devices of the present invention is not limited thereto. For example, multiple hosts H1 to H4 and multiple storage devices 110 to 130 can be replaced with other hosts or storage devices individually. Additional storage devices or additional hosts can be connected to a CXL (Compute Express Link) switch SW_CXL without changing the existing hardware configuration, so that the entire capacity of the storage system 100 can be increased, or additional functions can be provided.

[0038] refer to Figure 2 , the storage system 100 may include a plurality of hosts H1 to H4, a CXL switch SW_CXL, and a plurality of storage devices 110 to 130. In an embodiment, the storage system 100 may be a decomposed storage system. For example, the plurality of hosts H1 to H4 and the plurality of storage devices 110 to 130 may be connected to the CXL switch SW_CXL. Each of the plurality of hosts H1 to H4 may access the corresponding storage device 110 to 130 through the CXL switch SW_CXL.

[0039] In an embodiment, a plurality of hosts H1 to H4 may access or manage a plurality of storage devices 110 to 130 based on a plurality of metadata MD1 to MD4. For example, a first host H1 may access at least some of the plurality of storage devices 110 to 130 based on first metadata MD1. A second host H2 may access at least some of the plurality of storage devices 110 to 130 based on second metadata MD2. A third host H3 may access at least some of the plurality of storage devices 110 to 130 based on third metadata MD3. A fourth host H4 may access at least some of the plurality of storage devices 110 to 130 based on fourth metadata MD4.

[0040] In an embodiment, the metadata may include information about a storage area or memory area managed by the host. For example, the first metadata MD1 may include information about a memory physical address indicating a memory area (such as a system memory area) accessed by the first host H1. In this case, the memory physical address may be different from the physical address managed by the flash translation layer (FTL) of each storage device. For example, the memory physical address included in the first metadata MD1 of the first host H1 may be information indicating an area of ​​the system memory (e.g., multiple storage devices) of the first host H1. In contrast, the physical address managed by the FTL of the storage device may be information indicating a storage area of ​​a non-volatile memory device.

[0041] Alternatively, the first metadata MD1 may include a logical address indicating a storage space to be accessed by the first host H1. For example, the file system of the first host H1 may be configured to organize files or data used by the first host H1. The file system may manage storage areas of at least some of the storage devices 110 to 130 by using logical addresses. The file system may assign and manage logical addresses to data to be stored in the storage devices 110 to 130. As described above, each of the plurality of metadata MD1 to MD4 may be managed by the corresponding host H1 to H4, respectively. Depending on the manner implemented by each of the plurality of hosts H1 to H4, each of the metadata MD1 to MD4 may include information about a logical address managed by the file system or information about a physical address of a memory indicating a system memory area.

[0042] In the above-mentioned decomposed storage system, a specific storage device (eg, the first storage device 110) may be shared by at least two of the plurality of hosts H1 to H4. In this case, as described above, it may be necessary to maintain consistency of metadata MD1 to MD4 managed by the hosts H1 to H4.

[0043] According to an embodiment of the present disclosure, each of the plurality of storage devices 110 to 130 may detect changes in metadata MD1 to MD4 managed by hosts H1 to H4. Each of the plurality of storage devices 110 to 130 may send, broadcast, or multicast change information of the metadata MD1 to MD4 to the plurality of hosts H1 to H4. The plurality of hosts H1 to H4 may update the metadata MD1 to MD4 based on the received information so that the consistency of the metadata MD1 to MD4 is maintained. According to an embodiment of the present disclosure, a method for maintaining or managing the consistency of metadata will be described in detail later.

[0044] The first storage device 110 may include a controller 111, a nonvolatile memory device 112, and a dynamic random access memory (DRAM) device 113. In an embodiment, the first storage device 110 may be a storage device configured to comply with the CXL standard such as CXL storage, CXL memory module hybrid (CMM-H), or memory semantic solid state drive (MS-SSD). However, the present invention is not limited thereto.

[0045] The controller 111 may control the nonvolatile memory device 112 or the DRAM device 113 in response to a request received from the plurality of hosts H1 to H4 through the CXL switch SW_CXL. The controller 111 may include a CXL interface (I / F) circuit 111a, a flash translation layer (FTL) 111b, a hardware-level metadata consistency multicaster (HMCM) 111c (hereinafter referred to as a “consistency circuit (also referred to as a metadata consistency circuit)” for ease of description), a NAND interface (I / F) circuit 111d, and a DRAM controller 111e. Figure 2 The components included in the controller 111 are provided as examples, and the present invention is not limited thereto. For example, the controller 111 may also include various components such as a processor configured to control all operations of the controller 111, a buffer memory, an ECC engine, and an encryption engine.

[0046] The CXL interface circuit 111a may be connected to the CXL switch SW_CXL. The CXL interface circuit 111a may receive access requests issued from a plurality of hosts H1 to H4 through the CXL switch SW_CXL. In an embodiment, the access request may have a format conforming to CXL standards such as CXL.io and CXL.mem.

[0047] The FTL 111b can perform various operations that enable the non-volatile memory device 112 to be used efficiently, such as address mapping operations, wear leveling operations, and garbage collection operations. The address mapping operation refers to an operation of converting or mapping a logical address to a memory physical address. In this case, the logical address refers to an address or information managed by a plurality of hosts H1 to H4, and the memory physical address refers to information indicating the actual physical location of the non-volatile memory device 112 where data is stored. The wear leveling operation refers to an operation of allocating a plurality of storage blocks included in the non-volatile memory device 112 so that they are used evenly or balancing the erase counts of a plurality of storage blocks. Through the wear leveling operation, it is possible to prevent a specific storage block from being excessively degraded. The garbage collection operation refers to an operation of ensuring the available capacity of the non-volatile memory device 112 by copying valid data of a first storage block to a second storage block and then erasing the first storage block.

[0048] In an embodiment, the FTL 111b may be implemented by software or firmware. Information or program code associated with the FTL 111b may be stored in a buffer memory (not shown) of the controller 111, and may be driven by a processor (not shown) of the controller 111. In an embodiment, at least some or all of the functions of the FTL 111b may be implemented in the form of hardware, and at least some or all of the functions of the FTL 111b may be executed without the intervention of the processor.

[0049] The first consistency circuit 111c may include a consistency circuit. The consistency circuit may detect whether metadata MD1 to MD4 managed by multiple hosts H1 to H4 has changed based on (i.e., in response to) access requests received from multiple hosts H1 to H4. For example, when a request to write data in the first storage device 110 or to change the data of the first storage device 110 is received from the first host H1, the first metadata MD1 managed by the first host H1 may be changed. The first consistency circuit 111c (i.e., the consistency circuit) may detect changes in metadata caused by data write or change operations on the first storage device 110. The first consistency circuit 111c may send information about changes in metadata to any other host. In this case, the other hosts may be hosts that share the first storage device 110. In an embodiment, each other host may update its own metadata based on the change information. The operation of the first consistency circuit 111c will be described in detail later.

[0050] Under the control of the controller 111, the nonvolatile memory device 112 may store data or may output the stored data. In an embodiment, the nonvolatile memory device 112 may include a NAND flash memory (or referred to as a NAND flash memory device). However, the present invention is not limited thereto.

[0051] The NAND interface circuit 111d may control the nonvolatile memory device 112 so that data is stored in the nonvolatile memory device 112 or data is read from the nonvolatile memory device 112. In an embodiment, the NAND interface circuit 111d may include at least one of various flash memory interfaces such as a toggle NAND interface and an open NAND flash memory interface (ONFI). For example, the nonvolatile memory device 112 may include a plurality of NAND flash memory devices. In the case where the NAND interface circuit 111d is implemented based on the toggle interface, the NAND interface circuit 111d communicates with the plurality of NAND flash memory devices through a plurality of channels. The plurality of NAND flash memory devices may be connected to the plurality of channels in the form of a multi-channel (or multi-way) structure.

[0052] The NAND interface circuit 111d can send a chip enable signal / CE, a command latch enable signal CLE, an address latch enable signal ALE, a read enable signal / RE, and a write enable signal / WE to a plurality of NAND flash memory devices through a plurality of channels. The NAND interface circuit 111d and each of the plurality of NAND flash memory devices can exchange a data signal DQx and a data strobe signal DQS through a corresponding channel.

[0053] [Table 1]

[0054]

[0055] Table 1 shows the operation mode of the NAND flash memory device determined according to the signal state. Referring to Table 1, when the NAND flash memory device receives a command CMD or an address ADDR or receives / outputs (i.e., receives or outputs) data "DATA", the chip enable signal / CE maintains a low state "L". During the "command input" mode, the NAND interface circuit 111d can control the signal line so that the command latch enable signal CLE has a high level "H", the address latch enable signal ALE has a low level "L", the write enable signal / WE switches between a high level "H" and a low level "L", and the read enable signal / RE has a high level "H". During the "command input" mode, the NAND interface circuit 111d can send a command CMD to the NAND flash memory device through the data signal DQx in synchronization with the rising edge ↑ of the write enable signal / WE. The NAND flash memory device can recognize the command CMD in the data signal DQx in response to the rising edge ↑ of the write enable signal / WE. During the "address input" mode, the NAND interface circuit 111d can control the signal lines so that the command latch enable signal CLE has a low level "L", the address latch enable signal ALE has a high level "H", the write enable signal / WE switches between a high level "H" and a low level "L", and the read enable signal / RE has a high level "H". During the "address input" mode, the NAND interface circuit 111d can send an address ADDR to the NAND flash memory device through the data signal DQx in synchronization with the rising edge ↑ of the write enable signal / WE. The NAND flash memory device can recognize the address ADDR in the data signal DQx in response to the rising edge ↑ of the write enable signal / WE. In an embodiment, the address ADDR can be a value corresponding to the physical block address of the NAND flash memory device.

[0056] During the "data input" mode, the NAND interface circuit 111d may control the signal lines so that the command latch enable signal CLE has a low level "L", the address latch enable signal ALE has a low level "L", the write enable signal / WE has a high level "H", the read enable signal / RE has a high level "H", and the data strobe signal DQS switches between a high level "H" and a low level "L". During the "data input" mode, the NAND interface circuit 111d may send data "DATA" to the NAND flash memory device through the data signal DQx in synchronization with the rising edge ↑ and the falling edge ↓ of the data strobe signal DQS. The NAND flash memory device may identify the data "DATA" in the data signal DQx in response to the rising edge ↑ and the falling edge ↓ of the data strobe signal DQS.

[0057] During the "data output" mode, the NAND interface circuit 111d can control the signal lines so that the clock latch enable signal CLE has a low level "L", the address latch enable signal ALE has a low level "L", the write enable signal / WE has a high level "H", and the read enable signal / RE switches between a high level "H" and a low level "L". During the "data output" mode, the NAND flash memory device can generate a data strobe signal DQS that switches between a high level "H" and a low level "L" in response to the read enable signal / RE. The NAND flash memory device can send data "DATA" to the NAND interface circuit 111d through the data signal DQx in synchronization with the rising edge ↑ and the falling edge ↓ of the data strobe signal DQS. The NAND interface circuit 111d can identify the data "DATA" in the data signal DQx in response to the rising edge ↑ and the falling edge ↓ of the data strobe signal DQS.

[0058] The above-mentioned switching interfaces are examples, and the present invention is not limited thereto.

[0059] The DRAM controller 111e may be configured to store data in the DRAM device 113 or read data from the DRAM device 113. In an embodiment, the DRAM controller 111e may be implemented to comply with a standard protocol such as a DDR interface or a LPDDR interface.

[0060] The DRAM device 113 may operate under the control of the controller 111. In an embodiment, the DRAM device 113 may be configured to store various information necessary for the operation of the first storage device 110, such as mapping data to be used by the FTL 111b and information necessary for the operation of the controller 111. Alternatively, the DRAM device 113 may be a buffer memory configured to temporarily store write data (i.e., data to be stored) received from a plurality of hosts H1 to H4 or read data (i.e., data to be read) read from the nonvolatile memory device 112. Alternatively, the DRAM device 113 may be used or recognized as a system memory of the plurality of hosts H1 to H4.

[0061] In an embodiment, the first storage device 110 may be installed in or removed from a physical port based on a CXL interface (e.g., a PCIe physical port). For example, the first storage device 110 may have a connector of a storage system (so that the storage device 110 can be removably attached to and removed from a CXL switch (or a physical port based on a CXL interface). The connector may be mated with a corresponding connector of the CXL switch (e.g., mechanically plugged together to contact corresponding contacts of the connector to establish electrical signal and power connections therebetween). For example, in an embodiment, the first storage device 110 may be implemented based on an E1.S, E1.L, E3.S, E3.L, or PCIe AIC (CEM) form factor. Alternatively, the first storage device 110 may be implemented based on a U.2 form factor, an M.2 form factor, various different types of PCIe-based form factors, or various different types of small form factors. In an embodiment, the first storage device 110 may be implemented in various types of form factors and may provide a hot-swap function that enables the first storage device 110 to be installed in (or added to) or removed from a physical port or CXL switch SW_CXL.

[0062] In an embodiment, structures of the remaining storage devices 120 and 130 are similar to that of the first storage device 110 , and thus, repeated descriptions of the same, similar, or functionally equivalent elements (or operations) may not be given to avoid redundancy.

[0063] As described above in the example of the first consistency circuit 111c and the first storage device 110, according to an embodiment of the present invention, each of the plurality of storage devices 110 to 130 may include a consistency circuit 111c. The consistency circuit 111c of each of the plurality of storage devices 110 to 130 may include a consistency circuit. A consistency circuit (e.g., a consistency circuit) may detect changes in metadata MD1 to MD4 of a plurality of hosts H1 to H4. The consistency circuit of each of the plurality of storage devices 110 to 130 may send metadata change information to at least some of the plurality of hosts H1 to H4, and at least some of the plurality of hosts H1 to H4 may update their own metadata based on the change information. Therefore, the consistency of the metadata MD1 to MD4 may be maintained without separate communication between the plurality of hosts H1 to H4.

[0064] Figure 3A and Figure 3B Is used to describe Figure 2For ease of description, the access method of the storage system 100 will be described based on the hierarchical (or layer) structure of the first host H1 and the first storage device 110, and some components are omitted. However, the present invention is not limited thereto. For example, each of the plurality of hosts H1 to H4 and the plurality of storage devices 110 to 130 can be accessed by referring to Figure 3A and Figure 3B The described methods communicate with each other via the CXL switch SW_CXL.

[0065] First, refer to Figure 2 and Figure 3A , the hierarchical structure of the first host H1 may include an application APP, a memory management unit MMU, and a CXL driver DRV_CXL. The application APP may include various application programs driven by the first host H1.

[0066] The memory management unit MMU may be configured to manage the access of the application APP to the system memory based on the first metadata MD1. For example, the memory management unit MMU may be configured to translate the logical address of the data to be used by the application APP into the memory physical address PA of the system memory based on the first metadata MD1. The memory management unit MMU may issue an access request based on the memory physical address PA of the system memory.

[0067] The CXL driver DRV_CXL may receive an access request from the memory management unit MMU, and may convert the received access request to be suitable for the CXL standard. For example, based on the CXL.mem protocol as a memory access protocol, the CXL driver DRV_CXL may send an access request including a memory physical address PA to the first storage device 110. In an embodiment, in the CXL standard, the CXL.mem protocol may be a memory access protocol to be used for memory access.

[0068] The first storage device 110 may operate in response to an access request received from the first host H1. For example, the CXL interface circuit 111a of the first storage device 110 may receive a memory access request (e.g., CXL.mem) including a memory physical address PA from the first host H1. In an embodiment, the memory access request CXL.mem may be transmitted from the first host H1 to the CXL interface circuit 111a of the first storage device 110 through the CXL switch SW_CXL. In response to the memory access request CXL.mem received through the CXL interface circuit 111a, the first storage device 110 may access the DRAM device 113 to determine whether data or a memory area corresponding to the memory physical address PA exists in the DRAM device 113. In an embodiment, when the DRAM device 113 does not have data or a memory area corresponding to the memory physical address PA, the first storage device 110 may load data stored in the non-volatile memory device 112 to the DRAM device 113, or may refresh data stored in the DRAM device 113 to the non-volatile memory device 112, so that the memory area of ​​the DRAM device 113 is protected.

[0069] As described above, the memory management unit MMU of the first host H1 can manage the system memory area based on the first metadata MD1. In this case, the first host H1 can recognize the storage area of ​​the first storage device 110 as the system memory area, and the memory management unit MMU can manage the storage area of ​​the first storage device 110 as the system memory area. Therefore, the first host H1 can use the storage area of ​​the first storage device 110 as the system memory area, and can perform access to the first storage device 110 based on the memory access request CXL.mem.

[0070] for Figure 2 and Figure 3B , the hierarchical structure of the first host H1 may include an application APP, a file system FS, a memory management unit MMU, and a CXL driver DRV_CXL. The application APP, the memory management unit MMU, and the CXL driver DRV_CXL are described above, and therefore, repeated descriptions of the same, similar, or functionally equivalent elements (or operations) may not be given to avoid redundancy.

[0071] The file system FS may be configured to organize files or data used by the application APP. For example, the file system FS may manage the storage area of ​​the first storage device 110 by using the logical block address LBA. The file system FS may assign and manage the logical block address LBA to the data to be stored in the first storage device 110. In an embodiment, the type of the file system FS may be changed according to the operating system of the first host H1. In an embodiment, the file system FS may include at least one of the following various types of file systems: file allocation table (FAT), FAT32, NT file system (NTFS), hierarchical file system (HFS), journal file system (JSF2), XFS, on-disk structure-5 (ODS-5), UDF, ZFS, Unix file system (UFS), ext2, ext3, ext4, ReiserFS, Reiser4, ISO 9660, Gnome VFS, BFS and WinFS, or may include at least one of the following various types of distributed file systems: Google File System (GFS), Hadoop Distributed File System (HDFS), Andrew File System (AFS) and Gluster File System.

[0072] exist Figure 3B In the embodiment of the present invention, the first host H1 can be based on the reference Figure 3A The described method accesses the first storage device 110. For example, through the memory management unit MMU and the CXL driver DRV_CXL, the first host H1 may issue a memory access request CXL.mem including a memory physical address PA corresponding to the system memory, and may access the first storage device 110.

[0073] Alternatively, the first host H1 may access the first storage device 110 through the file system FS and the CXL driver DRV_CXL. For example, the application APP of the first host H1 may identify the logical block address LBA of the data or the storage space to be accessed through the file system FS. The CXL driver DRV_CXL may receive the logical block address LBA corresponding to the data or storage area to be accessed from the file system FS, and may issue an input / output (i.e., input or output) request CXL.io which is an input / output request protocol and includes the logical block address LBA. In an embodiment, the input / output request CXL.io may be a non-uniform input / output protocol based on PCI-e in the CXL standard.

[0074] The first storage device 110 may operate in response to a memory access request CXL.mem or an input / output request CXL.io from the first host H1. For example, the CXL interface circuit 111a of the first storage device 110 may receive the input / output request CXL.io from the first host H1. The first storage device 110 may access data or a storage area corresponding to a logical block address LBA in response to the input / output request CXL.io. In this case, the first storage device 110 may perform access to the nonvolatile memory device 112 through the FTL 111b in response to the input / output request CXL.io.

[0075] The CXL interface circuit 111a of the first storage device 110 may receive a memory access request CXL.mem from the first host H1. The first storage device 110 may perform access to the DRAM device 113 in response to the memory access request CXL.mem. In this case, the controller 111 of the first storage device 110 may operate as a cache controller. When the DRAM device 113 does not have data or a storage area corresponding to the memory physical address PA, the controller 111 (or cache controller) may perform access to the nonvolatile memory device 112.

[0076] In reference Figure 3A and Figure 3B In the access operation described above, the file system FS and the memory management unit MMU can perform the above management operation by using metadata. For example, when the first host H1 identifies the first storage device 110 as a system memory or manages the first storage device 110 as a system memory, the memory management unit MMU of the first host H1 can perform the above operation by using metadata including information about the memory physical address PA. When the first host H1 identifies the first storage device 110 as a storage medium, the file system FS of the first host H1 can perform the above operation by using metadata including the logical block address LBA.

[0077] That is, the first host H1 may recognize or manage the first storage device 110 as a system memory or a storage medium according to implementation. In this case, the metadata may include information about a logical block address LBA or a memory physical address PA.

[0078] As described above, the first host H1 may perform access to the first storage device 110 based on the input / output request CXL.io or the memory access request CXL.mem. For example, when the first host H1 recognizes the first storage device 110 as a system memory or manages the first storage device 110 as a system memory, the first host H1 may access the first storage device 110 through the memory access request CXL.memm. When the first host H1 recognizes the first storage device 110 as a storage medium, the first host H1 may access the first storage device 110 through the input / output request CXL.io.

[0079] Based on the operation description between the first host H1 and the first storage device 110 Figure 3A and Figure 3B For example, each of the plurality of hosts H1 to H4 may be based on a reference Figure 3A and Figure 3B The described access method communicates with one or more of the plurality of storage devices 110 - 130 .

[0080] In the following, the term "access request" is used to easily describe the embodiments of the present invention. An access request may be a request issued by a host to access a storage device. In this case, unless otherwise defined, an access request may refer to Figure 3A and Figure 3B The described input / output request CXL.io or memory access request CXL.mem. When the access request is the input / output request CXL.io, the access request may include a logical block address LBA, and the metadata of the relevant host may include information about the logical block address LBA. When the access request is the memory access request CXL.mem, the access request may include information about a memory physical address PA indicating a system memory area, and the metadata of the relevant host may include information about the memory physical address PA. The metadata may be included (i.e., stored) in the memory space of the corresponding host. Alternatively, the metadata may be included in the memory space of the storage device.

[0081] Figure 4 Is used to describe Figure 2 Figure 1.1 shows a diagram of storage areas that are recognized by multiple hosts. Figure 2 and Figure 4 , each of the plurality of hosts H1 to H4 may access the plurality of storage devices 110 to 130. For example, the plurality of hosts H1 to H4 may manage the storage areas SA1 to SA3 of the plurality of storage devices 110 to 130 by using the first to fourth metadata MD1 to MD4.

[0082] In an embodiment, each of the plurality of hosts H1 to H4 may share some of the storage areas SA1 to SA3 of the plurality of storage devices 110 to 130. For example, the first storage device 110 may include the first storage area SA1, the second storage device 120 may include the second storage area SA2, and the third storage device 130 may include the third storage area SA3. In this case, the first host H1 may access the storage area SA1 of the first storage device 110, the storage area SA2 of the second storage device 120, and the storage area SA3 of the third storage device 130. The second host H2 may access the storage area SA1 of the first storage device 110 and the storage area SA2 of the second storage device 120. The third host H3 may access the storage area SA1 of the first storage device 110, the storage area SA2 of the second storage device 120, and the storage area SA3 of the third storage device 130. The fourth host H4 may access the storage area SA2 of the second storage device 120 and the storage area SA3 of the third storage device 130.

[0083] In this case, the first metadata MD1 of the first host H1 may include information for managing the first storage area SA1, the second storage area SA2, and the third storage area SA3. The second metadata MD2 of the second host H2 may include information for managing the first storage area SA1 and the second storage area SA2. The third metadata MD3 of the third host H3 may include information for managing the first storage area SA1, the second storage area SA2, and the third storage area SA3, and the fourth metadata MD4 of the fourth host H4 may include information for managing the second storage area SA2 and the third storage area SA3.

[0084] In an embodiment, the information included in each of the first metadata MD1 to the fourth metadata MD4 may include a logical block address LBA or a memory physical address PA of the relevant storage area depending on how the relevant host is implemented (for example, depending on whether the storage area is managed as a storage medium or a system memory).

[0085] Figure 5 It's a picture. Figure 2 A flowchart of the operation of the first storage device. Figure 2 , Figure 4 and Figure 5 In operation S110, the first storage device 110 may receive an access request from the first host H1. In an embodiment, the access request may be a reference Figure 3A and Figure 3B Describes an input / output request CXL.io or a memory access request CXL.mem.

[0086] In operation S120, the first storage device 110 may perform an access operation corresponding to the received access request. For example, when the received access request is a data read request, the first storage device 110 may perform a read operation on the non-volatile memory device 112 or the DRAM device 113 based on the logical block address LBA or the memory physical address PA included in the received access request. When the received access request is a data write request, the first storage device 110 may perform a write operation on the non-volatile memory device 112 or the DRAM device 113 based on the logical block address LBA or the memory physical address PA included in the received access request. In an embodiment, when the received access request is a request for data movement or data copying, the first storage device 110 may perform an operation corresponding to data transfer or data copying.

[0087] In operation S130, the first storage device 110 may detect an update to the metadata. For example, when the received access request is a data write request, an update of the first metadata MD1 for the first host H1 may be performed. For example, the first host H1 may update information about a logical block address LBA or a memory physical address PA indicating a storage area where write data is to be stored in the first storage device 110.

[0088] The first consistency circuit 111c of the first storage device 110 may detect whether the first metadata MD1 is updated based on the access request received from the first host H1. For example, when the access request is a write request, the first consistency circuit 111c of the first storage device 110 may detect an update of the first metadata MD1 (i.e., detect any change in the first metadata MD1).

[0089] In operation S140, the first storage device 110 may multicast information about the update (ie, update information) to other hosts that share the first storage device 110. Figure 4 As illustrated, the first host H1, the second host H2, and the third host H3 may share the first storage area SA1 of the first storage device 110. The first host H1 may access the first storage area SA1 of the first storage device 110 and may update the first metadata MD1 of the first host H1. Based on an access request from the first host H1, the first storage device 110 may detect any update to the first metadata (i.e., whether the first metadata MD1 has changed). The first storage device 110 may multicast or send update information of the first metadata MD1 to the second host H2 and the third host H3.

[0090] In an embodiment, the update information of the first metadata MD1 may include information about, for example, information about a logical block address LBA or a memory physical address PA. In an embodiment, the second host H2 and the third host H3 may update the second metadata MD2 and the third metadata MD3 by using the provided update information. Therefore, the consistency of the first metadata MD1 to the third metadata MD3 of the first host H1 to the third host H3 sharing the first storage device 110 may be maintained. Alternatively, the consistency of information associated with the first storage area SA1 of the first storage device 110 may be maintained between the first metadata MD1 to the third metadata MD3 of the first host H1 to the third host H3.

[0091] Figure 6 It is a diagram of Figure 2 FIG. 1 is a diagram of a shared table used by a multicast circuit of a first storage device. Figure 5 As described, the first storage device 110 may be shared by the first host H1, the second host H2, and the third host H3. In this case, the sharing table ST of the first storage device 110 may include information about the first host H1, the second host H2, and the third host H3.

[0092] For example, the sharing table ST of the first storage device 110 may include the port identifiers PID_H1, PID_H2, and PID_H3 of the first host H1, the second host H2, and the third host H3 and the callback parameters cbp_H1, cbp_H2, and cbp_H3. Fig. 7A ) can be sent or multicast from the first storage device 110 to the first host H1, the second host H2 and the third host H3 through (i.e., by using) the port identifiers PID_H1, PID_H2 and PID_H3 of the first host H1, the second host H2 and the third host H3 and the callback parameters cbp_H1, cbp_H2 and cbp_H3.

[0093] In an embodiment, the sharing table ST may be created or updated in the initialization operation of the first storage device 110. In the initialization operation of the first storage device 110, the sharing table ST may be created based on information about which hosts share the first storage device 110. For example, the structure (or architecture) of the storage system 100 may be changed so that a new host is added, an existing host is replaced with a new host, or an existing host is removed. When those kinds of changes occur, the sharing table ST may be updated.

[0094] Fig. 7A and Figure 7B Is used to describe Figure 2 For the sake of simplicity of the drawings, some components will be omitted, but the present invention is not limited thereto.

[0095] First, if Fig. 7A As shown, the first host H1 may store data in a storage area corresponding to a first address ADDR1. In an embodiment, the first address ADDR1 may indicate an area belonging to a first storage area SA1 of the first storage device 110. The first host H1 may send an access request RQ_ACC including the first address ADDR1 to the first storage device 110 through the CXL switch SW_CXL. In an embodiment, the first address ADDR1 may be in the form of a logical block address LBA or a memory physical address PA. In an embodiment, the access request RQ_ACC may be an input / output request CXL.io or a memory access request CXL.mem.

[0096] The CXL interface circuit 111a of the first storage device 110 may receive an access request RQ_ACC including the first address ADDR1 from the first host H1 through the CXL switch SW_CXL. The first storage device 110 may write write data into an area corresponding to the first storage area SA1 in the first storage device 110 in response to the received access request RQ_ACC.

[0097] In an embodiment, when the first host H1 writes write data in the first storage area SA1 of the first storage device 110 , the first host H1 may update information corresponding to the first address ADDR1 in the first metadata MD1 .

[0098] Next, if Figure 7B As illustrated, the first consistency circuit 111c of the first storage device 110 may detect that the first metadata MD1 of the first host H1 is updated based on the access request RQ_ACC received from the first host H1. The first consistency circuit 111c may send or multicast update information INF_u to the hosts included in the shared table ST. In an embodiment, the update information INF_u may include information about the first address ADDR1 updated in the first metadata MD1.

[0099] The second host H2 and the third host H3 that share the first storage device 110 with the first host H1 may receive the update information INF_u through the CXL switch SW_CXL. The second host H2 and the third host H3 may respectively update the second metadata MD2 and the third metadata MD3 based on the update information INF_u. For example, the update information INF_u may include information about the first address ADDR1 and size information. Alternatively, the second host H2 may update a portion of the information of the second metadata MD2 based on (i.e., by using) the update information INF_u including the first address ADDR1. A portion of the information of the second metadata MD2 may correspond to the first address information ADDR1. The third host H3 may update the information of the third metadata MD3 based on the update information INF_u, and the information of the third metadata MD3 corresponds to the first address ADDR1.

[0100] Therefore, the second host H2 and the third host H3 can update the second metadata MD2 and the third metadata MD3 without separate communication between the first host H1 to the third host H3 by using the update information INF_u received from the first storage device 110. As a result, the consistency of the first metadata MD1 to the third metadata MD3 of the first host H1 to the third host H3 can be maintained without separate communication between the first host H1 to the third host H3, thereby reducing the overhead of inter-host communication between the hosts and ultimately improving the performance of the entire storage system.

[0101] Figure 8 It is used to describe how to update Figure 2 As described above, the update information INF_u about the first metadata MD1 of the first host H1 may be transmitted or multicasted from the first storage device 110 to the second host H2. In this case, the second host H2 may update the second metadata MD2 based on the update information INF_u.

[0102] For example, the second host H2 may include an interrupt handler 101a, a metadata manager 101b, and second metadata MD2. In an embodiment, the update information INF_u sent or multicast from the first storage device 110 may be provided in an interrupt manner through the CXL switch SW_CXL. For example, when the update information INF_u is sent from the first storage device 110 to the CXL switch SW_CXL, the CXL switch SW_CXL may send an interrupt signal to the second host H2. Alternatively, the first storage device 110 may send an interrupt signal for sending the update information INF_u to the second host H2 through the CXL switch SW_CXL.

[0103] The interrupt handler 101a of the second host H2 may transmit the update information INF_u to the metadata manager 101b in response to the interrupt signal. Alternatively, the interrupt handler 101a of the second host H2 may activate the metadata manager 101b in response to the interrupt signal, and the metadata manager 101b thus activated may receive the update information INF_u through the CXL switch SW_CXL.

[0104] The metadata manager 101b may update the second metadata MD2 based on the received update information INF_u. For example, the metadata manager 101b may update a portion of the information of the second metadata MD2. The portion of the information of the second metadata MD2 may be information corresponding to an address included in the update information INF_u. In an embodiment, the metadata manager 101b may be implemented in the form of software, firmware, hardware, or a combination thereof.

[0105] In an embodiment, the update information INF_u may include area information (e.g., ADDR1) of metadata (e.g., MD1) updated by any other host (e.g., H1). In this case, the area information (e.g., ADDR1) may be in the form of a logical block address LBA or a memory physical address PA. The metadata manager 101b may provide a function (e.g., inverse_filemap) of searching for an area (e.g., a file system area or a system memory area) of metadata corresponding to an address included in the update information.

[0106] refer to Figure 8 The configuration and update operation of the described interruption mode may be an exemplary embodiment according to the present invention, but the present invention is not limited thereto.

[0107] For example, each of the plurality of storage devices 110 to 130 may send or multicast update information INF_u to the plurality of hosts H1 to H4 by using various types of responses (e.g., S2M NDR (No Data Response) and S2M DRS (Data Response)) and various types of formats (e.g., D2H and S2M formats) supported by the CXL standard.

[0108] Fig. 9 The reference is shown in the figure Fig. 7A and Figure 7B A flowchart describing the operation of the storage system. Figure 2 , Fig. 7A , Figure 7B and Fig. 9 In operation S210, the first host H1 may send an access request RQ_ACC to the CXL switch SW_CXL. Fig. 7AAs described, the access request RQ_ACC may be a write request for the area corresponding to the first address ADDR1.

[0109] In operation S211 , the CXL switch SW_CXL may transmit an access request RQ_ACC to the first storage device 110 .

[0110] In operation S220, the first storage device 110 may perform an access operation corresponding to the access request RQ_ACC received from the CXL switch SW_CXL. Fig. 7A As described, the first memory device 110 may write write data in a region corresponding to the first address ADDR1 in response to the access request RQ_ACC.

[0111] In operation S230, the first storage device 110 may send an access response RSP_ACC to the access request RQ_ACC to the CXL switch SW_CXL. In operation S231, the CXL switch SW_CXL may transmit the access response RSP_ACC to the first host H1. The first host H1 may recognize that the operation corresponding to the access request RQ_ACC is completed based on the access response RSP_ACC.

[0112] In operation S240, the first storage device 110 may determine whether the metadata is updated or changed. Figure 7B As illustrated, the first coherency circuit 111 c of the first storage device 110 may determine whether the first metadata MD1 is updated or changed based on the access request RQ_ACC received from the first host H1 .

[0113] When it is determined that there is no update of the first metadata MD1, the first storage device 110 may not perform any further operation. When it is determined that the first metadata MD1 is updated, in operation S250, the first storage device 110 may send update information INF_u to the CXL switch SW_CXL. Figure 7B As described, the update information INF_u may include information about the first address ADDR1 and size information. In an embodiment, based on the shared table ST, the first storage device 110 may transmit the update information INF_u together with the second port identifier PID2 / second callback parameter cbp_H2 of the second host H2 and the third port identifier PID3 / third callback parameter cbp_H3 of the third host H3 to the CXL switch SW_CXL.

[0114] In operation S251, the CXL switch SW_CXL may transmit update information INF_u to the second host H2. In operation S252, the CXL switch SW_CXL may transmit update information INF_u to the third host H3. The CXL switch SW_CXL may send the update information INF_u to each of the second host H2 and the third host H3 based on information (e.g., PID2 / cbp_H2 and PID3 / cbp_H3) received together with the update information INF_u. For example, the CXL switch SW_CXL may send the update information INF_u based on PID2 / cbp_H2 (i.e., PID2 and cbp_H2) and PID3 / cbp_H3.

[0115] In an embodiment, operations S250 to S252 may constitute a multicast operation. A multicast operation refers to a method for simultaneously sending data or information from one sender to multiple receivers. Unlike unicast, multicast enables a sender to efficiently send data or information to a selected group of receivers.

[0116] In operation S261, the second host H2 may update the second metadata MD2 based on the update information INF_u. In operation S262, the third host H3 may update the third metadata MD3 based on the update information INF_u. Figure 7B and Figure 8 Operation S261 and operation S262 are described, so repeated description of the same, similar, or functionally equivalent elements (or operations) may not be given to avoid redundancy.

[0117] In an embodiment of the present invention, a set of operations of the system 100 may be repeatedly performed, such as the following set of operations. For example, the first storage device 110 may receive an access request from the second host H2 through the CXL switch SW_CXL. The first storage device 110 may perform an operation corresponding to the access request from the second host H2. The first storage device 110 may detect a change in the second metadata MD2 managed by the second host H2 based on the access request from the second host H2. The first storage device 110 may send update information corresponding to the change in the second metadata MD2 to the first host H1 through the CXL switch SW_CXL. The first host H1 and the second host H2 may share the first storage device 110.

[0118] FIG. 10A to FIG. 10C Is used to describe Figure 2 For the sake of simplicity of the drawings and for the convenience of description, some components are omitted in the drawings, and descriptions associated with the above components are omitted.

[0119] like Fig. 10A and Fig. 10BAs illustrated, the first host H1 may perform an operation of copying first data stored in an area of ​​a first address ADDR1 to an area of ​​a second address ADDR2. In an embodiment, the area of ​​the first address ADDR1 may be included in a first storage area SA1 of the first storage device 110, and the area of ​​the second address ADDR2 may be included in a second storage area SA2 of the second storage device 120. The first data stored in the first storage area SA1 may be copied by the first host H1 to the second storage area SA2. As the first data stored in the area of ​​the first address ADDR1 is copied to the area of ​​the second address ADDR2, the first host H1 may update the first metadata MD1.

[0120] For example, refer to Fig. 10A , the first host H1 may send a replication request RQ_REP to the first storage device 110 and the second storage device 120 through the CXL switch SW_CXL. In an embodiment, the replication request RQ_REP may include information about a first address ADDR1 (or source address) indicating an area where the first data as a replication target is stored and a second address ADDR2 (or target address) indicating an area where the first data is to be copied. In an embodiment, each of the first address ADDR1 and the second address ADDR2 may be in the form of a logical block address LBA or a memory physical address PA.

[0121] refer to Fig. 10B , the first storage device 110 and the second storage device 120 may perform a copy operation associated with the first data in response to the copy request RQ_REP received from the first host H1. For example, the first storage device 110 may send the first data stored in the area of ​​the first address ADDR1 to the CXL switch SW_CXL through the CXL interface circuit 111a in response to the copy request RQ_REP. The CXL interface circuit 121a of the second storage device 120 may receive the first data from the CXL switch SW_CXL. The second storage device 120 may write the first data in the area of ​​the second address ADDR2 included in the copy request RQ_REP. Therefore, the first storage device 110 may send the first data stored in the first storage device 110 to the external storage device (i.e., the second storage device 120) through the CXL switch SW_CXL in response to the copy request RQ_REP.

[0122] As described above, the first storage device 110 and the second storage device 120 may exchange data with each other through the CXL switch SW_CXL. In an embodiment, the first storage device 110 and the second storage device 120 may exchange data with each other in a P2P manner without intervention of the hosts H1 to H4. In an embodiment, the first storage device 110 and the second storage device 120 may exchange data with each other through the CXL switch SW_CXL based on the unordered IO (UIO) protocol. In an embodiment, the first storage device 110 and the second storage device 120 may exchange data with each other through an input / output request CXL.io or a memory access request CXL.mem.

[0123] refer to Fig. 10C , the second consistency circuit 121c of the second storage device 120 can detect the update of the first metadata MD1. For example, the second consistency circuit 121c of the second storage device 120 can detect the update of the first metadata MD1 based on the replication request RQ_REP. The information of the first metadata MD1 associated with the area of ​​the second address ADDR2 can be updated. Therefore, the second consistency circuit 121c can send or multicast the update information INF_u to the second host H2 to the fourth host H4 that share the second storage area SA2. In an embodiment, the update information INF_u may include information about the first address ADDR1 as a source address and the second address ADDR2 as a target address.

[0124] The second host H2, the third host H3, and the fourth host H4 may receive the update information INF_u through the CXL switch SW_CXL, and may update the second metadata MD2, the third metadata MD3, and the fourth metadata MD4 based on the received update information INF_u. Therefore, the consistency of the second storage area SA2 may be maintained between the first metadata MD1 to the fourth metadata MD4 respectively managed by the first host H1 to the fourth host H4.

[0125] Fig.11 The reference is shown in the figure FIG. 10A to FIG. 10C Flowchart of the operation of the storage system described. For the convenience of description, the description associated with the above-mentioned components will be omitted to avoid redundancy.

[0126] refer to Figure 2 and FIG. 10A to FIG. 11 In operation S310, the first host H1 may send a replication request RQ_REP to the CXL switch SW_CXL. In an embodiment, the replication request RQ_REP may include a first address ADDR1 corresponding to the first storage area SA1 and a second address ADDR2 corresponding to the second storage area SA2. The first address ADDR1 may be a source address, and the second address ADDR2 may be a target address.

[0127] In operation S311, the CXL switch SW_CXL may transmit a replication request RQ_REP to the first storage device 110. In operation S312, the CXL switch SW_CXL may transmit the replication request RQ_REP to the second storage device 120. In operation S320, the first storage device 110 and the second storage device 120 may perform a data replication operation in response to the replication request RQ_REP. For example, the first storage device 110 may transmit first data stored in an area of ​​the first storage device 110 to the second storage device 120 through the CXL switch SW_CXL, and the second storage device 120 may store the first data in an area corresponding to the second address ADDR2.

[0128] In operation S330, the second storage device 120 may send a replication response RSP_REP to the CXL switch SW_CXL. In operation S331, the CXL switch SW_CXL may send a replication response RSP_REP to the first host H1. In an embodiment, although Fig.11 Although not shown in the figure, the first storage device 110 may send a replication response RSP_REP to the first host 11 through the CXL switch SW_CXL. The first host H1 may recognize that the replication operation is completed based on the replication response RSP_REP.

[0129] In operation S340, the second storage device 120 may send update information INF_u to the CXL switch SW_CXL. In operation S341, the CXL switch SW_CXL may send update information INF_u to the second host H2. In operation S342, the CXL switch SW_CXL may send update information INF_u to the third host H3. In operation S343, the CXL switch SW_CXL may send update information INF_u to the fourth host H4.

[0130] In an embodiment, the sharing table ST of the second storage device 120 may include information (e.g., port identifiers and callback parameters) about the hosts H1, H2, H3, and H4 that share the second storage device 120. Based on the sharing table ST, the second storage device 120 may transmit the update information INF_u together with the second port identifier PID2 / second callback parameter cbp_H2 of the second host H2, the third port identifier PID3 / third callback parameter cbp_H3 of the third host H3, and the fourth port identifier PID4 / fourth callback parameter cbp_H4 of the fourth host H4 to the CXL switch SW_CXL. The CXL switch SW_CXL may send the update information INF_u to each of the hosts H2, H3, and H4 based on the information (e.g., port identifiers and callback parameters) received together with the update information INF_u. In an embodiment, operations S340 to S343 may constitute a multicast operation for the update information INF_u.

[0131] In operation S351, the second host H2 may update the second metadata MD2 based on the update information INF_u. In operation S352, the third host H3 may update the third metadata MD3 based on the update information INF_u. In operation S353, the fourth host H4 may update the fourth metadata MD4 based on the update information INF_u.

[0132] Fig.12 It's a picture. Figure 2 A diagram of a shared table included in a first storage device. Figure 6 and Fig.12 The differences between embodiments of may lie in the nature of the information in the shared table. Figure 6 In the example, the sharing table ST of the first storage device 110 only includes information about which host the first storage device 110 is shared by. In contrast, the sharing table ST-1 of the first storage device 110 may include information about which host each of the first storage device 110 to the third storage device 130 is shared by.

[0133] In reference Figure 6 In the depicted embodiment, the sharing table ST of the first storage device 110 includes hosts H1, H2, and H3 that share the first storage device 110 (ie, itself).

[0134] In contrast, Fig.12In an embodiment of the present invention, the sharing table ST-1 of the first storage device 110 may include information about which host among the hosts H1 to H4 shares each of the first to third storage devices 110 to 130. For example, the sharing table ST-1 may include information PID1 / cbp_H1, PID2 / cbp_H2, and PID3 / cbp_H3 about the first host H1, the second host H2, and the third host H3 sharing the first storage device 110, may include information PID1 / cbp_H1, PID2 / cbp_H2, PID3 / cbp_H3, and PID4 / cbp_H4 about the first host H1, the second host H2, the third host H3, and the fourth host H4 sharing the second storage device 120, and may include information PID1 / cbp_H1, PID3 / cbp_H3, and PID4 / cbp_H4 about the first host H1, the third host H3, and the fourth host H4 sharing the third storage device 130.

[0135] In an embodiment, when the sharing table ST-1 of the first storage device 110 includes information about the hosts H1 to H4 that share any other storage devices (e.g., 120 and 130), the first storage device 110 may detect updates of metadata corresponding to the storage devices 120 and 130, and may send or multicast update information to the hosts H1 to H4, as described below in conjunction with Fig.13 As described. Fig.13 This embodiment is different from Fig. 10C The second consistency circuit 121c of the second storage device 120 sends or multicasts the update information to the relevant hosts.

[0136] Fig.13 Is used to describe Figure 2 For ease of description, descriptions associated with the above components will be omitted to avoid redundancy. Figure 1 , Fig.12 and Fig.13 , data included in the first storage device 110 may be copied to the second storage device 120 by the first host H1. In this case, the first host H1 may update first metadata MD1 corresponding to the second storage area SA2 of the second storage device 120.

[0137] In this case, based on the sharing table ST-1, the first storage device 110 may send update information INF_u to the second host H2, the third host H3, and the fourth host H4 that share the second storage area SA2 of the second storage device 120. In this case, the update information INF_u may include the first address ADDR1 as a source address and the second address ADDR2 as a target address.

[0138] The second host H2 , the third host H3 , and the fourth host H4 may update second metadata MD2 , third metadata MD3 , and fourth metadata MD4 corresponding to the second storage area SA2 based on the update information INF_u received from the first storage device 110 through the CXL switch SW_CXL.

[0139] Fig.14 It is a diagram showing the Fig.13 Flowchart of the operation of the storage system of an embodiment. Figure 2 and Figure 12 to Figure 14 , the first host H1, the CXL switch SW_CXL, the first storage device 110 and the second storage device 120 may perform operations S410 to S431. Operations S410 to S431 are similar to Fig.11 Operation S310 to operation S331 of the present invention are described above, and thus, repeated description of the same, similar, or functionally equivalent elements (or operations) may not be given to avoid redundancy.

[0140] In operation S440, the first storage device 110 may send update information INF_u to the CXL switch SW_CXL. For example, based on the replication request RQ_REP in operation S411, the first storage device 110 may detect that metadata corresponding to the second storage area SA2 of the second storage device 120 is updated. In this case, the first storage device 110 may send update information INF_u of the metadata to the CXL switch SW_CXL. In an embodiment, based on the sharing table ST-1, the first storage device 110 may send information PID2 / cbp_H2, PID3 / cbp_H3, and PID4 / cbp_H4 about the hosts H2, H3, and H4 sharing the second storage device 120 together with the update information INF_u to the CXL switch SW_CXL.

[0141] Thereafter, the CXL switch SW_CXL and the second to fourth hosts H2 to H4 may perform operations S441 to S453. Operations S441 to S453 are similar to Fig.11 Operations S341 to S353 are described in detail, and therefore, repeated descriptions of the same, similar, or functionally equivalent elements (or operations) may not be given to avoid redundancy.

[0142] As described above, according to an embodiment of the present invention, the first host H1 may control data replication between the first storage device 110 and the second storage device 120 connected to the CXL switch SW_CXL. In this case, the first storage device 110 and the second storage device 120 may exchange data with each other in a P2P manner. Also, at least one of the first storage device 110 and the second storage device 120 may detect an update of metadata, and may multicast the detected update information INF_u to a host corresponding thereto.

[0143] FIG. 15A to FIG. 15D Is used to describe Figure 2 For the sake of simplicity of the drawings and for the convenience of description, some components are omitted in the drawings, and descriptions associated with the above components are omitted.

[0144] First, if Fig.15A As illustrated, the first host H1 may perform an operation of moving the first data stored in the region of the first address ADDR1 to the region of the third address ADDR3. In an embodiment, the region corresponding to the first address ADDR1 may be included in the first storage area SA1 of the first storage device 110, and the region corresponding to the third address ADDR3 may be included in the third storage area SA3 of the third storage device 130. For example, the first data stored in the first storage area SA1 may be moved by the first host H1 to the third storage area SA3.

[0145] In this case, the first host H1 may send a move request RQ_MV to the first storage device 110 and the third storage device 130 through the CXL switch SW_CXL. In an embodiment, the move request RQ_MV may include information about a first address ADDR1 (or source address) indicating an area where the first data as a move target is stored and a third address ADDR3 (or target address) indicating an area where the first data is to be moved. In an embodiment, each of the first address ADDR1 and the third address ADDR3 may be in the form of a logical block address LBA or a memory physical address PA.

[0146] When the first data stored in the area of ​​the first address ADDR1 is moved to the area of ​​the third address ADDR3, the first host H1 may update the first metadata MD1. For example, the first host H1 may update the first metadata MD1 so that the information of the area corresponding to the first address ADDR1 is deleted or invalidated, and the information of the area corresponding to the third address ADDR3 is re-added.

[0147] Next, refer to Fig. 15B, the first storage device 110 and the third storage device 130 may perform a move operation associated with the first data in response to the move request RQ_MV. For example, the first storage device 110 may send the first data stored in the area of ​​the first address ADDR1 to the CXL switch SW_CXL through the CXL interface circuit 111a in response to the move request RQ_MV. The CXL interface circuit 131a of the third storage device 130 may receive the first data from the CXL switch SW_CXL. The third storage device 130 may write the first data in the area of ​​the third address ADDR3 included in the move request RQ_MV. The first storage device 110 may delete the first data stored in the area of ​​the first address ADDR1.

[0148] As described above, the first storage device 110 and the third storage device 130 can exchange data with each other through the CXL switch SW_CXL. In this case, the data transmission between the first storage device 110 and the third storage device 130 is similar to the reference Fig. 10B The data transmission described herein is simple, and therefore, repeated description of the same, similar, or functionally equivalent elements (or operations) may not be given to avoid redundancy.

[0149] Then, refer to Fig. 15C , the first consistency circuit 111c of the first storage device 110 may detect the update of the first metadata MD1. For example, based on the move request RQ_MV, the first consistency circuit 111c of the first storage device 110 may detect that information about the area corresponding to the first address ADDR1 is updated in the first metadata MD1 of the first host H1.

[0150] In response to detecting the update of the first metadata MD1, the first consistency circuit 111c of the first storage device 110 may multicast the first update information INF_u1 about the first address ADDR1. For example, this may be performed based on the shared table ST (refer to Figure 6 ), checks information about the second host H2 and the third host H3 that share the first storage area SA1 of the first storage device 110 (i.e., PID2 / cbp_H2 and PID3 / cbp_H3). The first consistency circuit 111c of the first storage device 110 may multicast the first update information INF_u1 to the second host H2 and the third host H3 through the CXL switch SW_CXL based on the information PID2 / cbp_H2 and PID3 / cbp_H3 about the second host H2 and the third host H3. In an embodiment, the first update information INF_u1 may include information about the first address ADDR1 corresponding to the first storage area SA1 of the first storage device 110.

[0151] The second host H2 and the third host H3 may update the second metadata MD2 and the third metadata MD3, respectively, based on the first update information INF_u1 received through the CXL switch SW_CXL. For example, the second host H2 may update the second metadata MD2 so that the information of the area corresponding to the first address ADDR1 is deleted or invalidated, and the third host H3 may update the third metadata MD3 so that the information of the area corresponding to the first address ADDR1 is deleted or invalidated.

[0152] Then, refer to Fig.15D , the third consistency circuit 131c of the third storage device 130 may detect the update of the first metadata MD1. For example, based on the move request RQ_MV, the third consistency circuit 131c of the third storage device 130 may detect that information about the area corresponding to the third address ADDR3 is updated in the first metadata MD1 of the first host H1.

[0153] In response to detecting the update of the first metadata MD1, the third consistency circuit 131c of the third storage device 130 may multicast the third update information INF_u3 about the third address ADDR3. Figure 4 As described, the third storage device 130 may be shared by the first host H1, the third host H3, and the fourth host H4. In this case, the sharing table included in the third storage device 130 may include information PID1 / cbp_H1, PID3 / cbp_H3, and PID4 / cbp_H4 about the first host H1, the third host H3, and the fourth host H4. The third storage device 130 may check information PID3 / cbp_H3 and PID4 / cbp_H4 about the third host H3 and the fourth host H4 sharing the third storage area SA3 of the third storage device 130 based on the sharing table. The third storage device 130 may multicast the third update information INF_u3 to the third host H3 and the fourth host H4 through the CXL switch SW_CXL based on the information PID3 / cbp_H3 and PID4 / cbp_H4 about the third host H3 and the fourth host H4. In an embodiment, the third update information INF_u3 may include information of a third address ADDR3 corresponding to the third storage area SA3 of the third storage device 130.

[0154] The third host H3 and the fourth host H4 may respectively update the third metadata MD3 and the fourth metadata MD4 based on the third update information INF_u3 received through the CXL switch SW_CXL. For example, the third host H3 may update the third metadata MD3 so that the information of the area corresponding to the third address ADDR3 is changed, and the fourth host H4 may update the fourth metadata MD4 so that the information of the area corresponding to the third address ADDR3 is changed. As a result, consistency may be maintained between the first metadata MD1 to the fourth metadata MD4 of the first host H1 to the fourth host H4.

[0155] Fig.16 It's a picture. FIG. 15A to FIG. 15D For ease of description, descriptions associated with the above components will be omitted to avoid redundancy. Figure 2 and FIG. 15A to FIG. 16 In operation S510, the first host H1 may send a move request RQ_MV to the CXL switch SW_CXL. In operation S511, the CXL switch SW_CXL may transmit the move request RQ_MV to the first storage device 110. In operation S512, the CXL switch SW_CXL may transmit the move request RQ_MV to the third storage device 130. In operation S520, the first storage device 110 and the third storage device 130 may perform a move operation associated with the first data through the CXL switch SW_CXL in response to the move request RQ_MV.

[0156] In operation S530 , the third storage device 130 may send a response RSP_MV to the move request RQ_MV to the CXL switch SW_CXL. In operation S531 , the CXL switch SW_CXL may transmit a response RSP_MV to the move request RQ_MV to the first host H1 .

[0157] In operation S540 , the first storage device 110 may transmit first update information INF_u1 to the CXL switch SW_CXL. The first update information INF_u1 may include information on an update of metadata corresponding to the first storage area SA1 of the first storage device 110 .

[0158] In an embodiment, the first storage device 110 may be shared by the first host H1, the second host H2, the third host H3, and the fourth host H4. Information of the first metadata MD1 associated with the first storage area SA1 may be updated by the first host H1, and therefore, the first storage device 110 may multicast the first update information INF_u1 to the remaining hosts H2, H3, and H4. To this end, the first storage device 110 may send information PID2 / cbp_H2, PID3 / cbp_H3, and PID4 / cbp_H4 about the remaining hosts H2, H3, and H4 together with the first update information INF_u1 to the CXL switch SW_CXL.

[0159] In operation S541, the CXL switch SW_CXL may transmit the first update information INF_u1 to the second host H2. In operation S542, the CXL switch SW_CXL may transmit the first update information INF_u1 to the third host H3. In operation S543, the CXL switch SW_CXL may transmit the first update information INF_u1 to the fourth host H4.

[0160] In operation S550 , the third storage device 130 may transmit third update information INF_u3 to the CXL switch SW_CXL. The third update information INF_u3 may include updated information on metadata corresponding to the third storage area SA3 of the third storage device 130 .

[0161] In an embodiment, the third storage device 130 may be shared by the first host H1, the third host H3, and the fourth host H4. Information of the first metadata MD1 associated with the third storage area SA3 may be updated by the first host H1, and therefore, the third storage device 130 may multicast third update information INF_u3 to the remaining hosts H3 and H4. To this end, the third storage device 130 may send information PID3 / cbp_H3 and PID4 / cbp_H4 about the remaining hosts H3 and H4 together with the third update information INF_u3 to the CXL switch SW_CXL.

[0162] In operation S551, the CXL switch SW_CXL may transmit the third update information INF_u3 to the third host H3. In operation S552, the CXL switch SW_CXL may transmit the third update information INF_u3 to the fourth host H4.

[0163] In operation 561, the second host H2 may update the second metadata MD2 based on the first update information INF_u1. In operation 562, the third host H3 may update the third metadata MD3 based on the first update information INF_u1 and the third update information INF_u3. In operation S563, the fourth host H4 may update the fourth metadata MD4 based on the first update information INF_u1 and the third update information INF_u3.

[0164] As described above, each of the plurality of storage devices 110, 120, and 130 may be shared by a plurality of hosts H1, H2, H3, and H4. In this case, each of the plurality of storage devices 110, 120, and 130 may detect changes or updates to metadata corresponding to its storage area, and may multicast update information to any other hosts that share each of the plurality of storage devices 110, 120, and 130. Each of the plurality of hosts H1, H2, H3, and H4 may update metadata based on the received update information. Therefore, without separate communication or protocols between the plurality of hosts H1, H2, H3, and H4, consistency may be maintained between a plurality of metadata respectively managed by the plurality of hosts H1, H2, H3, and H4.

[0165] Fig.17 is a block diagram illustrating a storage system according to an embodiment of the present invention. Fig.17 , the storage system 200 may include a plurality of hosts H1, H2, H3, and H4, a CXL switch SW_CXL, a plurality of storage devices 210, 220, and 230, and a consistency device 201. The consistency device 201 may include a metadata consistency circuit 111c as described herein, and thus, a repeated description thereof may be omitted to avoid redundancy. In addition, the consistency device 201 may include its own Figure 2 The CXL interface circuit 111a is the same as the CXL interface circuit 111a of the embodiment. Through (ie, by using) the CXL interface circuit, the consistency device 201 can communicate with the CXL switch SW_CXL to perform the following operations.

[0166] As in reference Figure 2 As in the above description given, the plurality of hosts H1, H2, H3, and H4 may access the plurality of storage devices 210, 220, and 230 through the CXL switch SW_CXL. In this case, the plurality of hosts H1, H2, H3, and H4 may manage storage areas of the plurality of storage devices 210, 220, and 230 based on the first to fourth metadata MD1, MD2, MD3, and MD4.

[0167] As in the above description, multiple hosts H1, H2, H3, and H4 may share multiple storage devices 210, 220, and 230. In this case, consistency device 201 may detect any changes to metadata made by multiple hosts H1 to H4 based on each access request issued by multiple hosts H1 to H4 to access multiple storage devices 210, 220, and 230. When an update to the metadata is detected, consistency device 201 may send update information to the corresponding host.

[0168] For example, the first host H1 may issue a first access request to access the first storage area SA1 of the first storage device 210. The first storage device 210 may perform access to the first storage area SA1 in response to the first access request. Therefore, the first host H1 may update the first metadata MD1 so that the information about the first storage area SA1 is changed. The consistency device 201 may monitor or check the first access request through the CXL switch SW_CXL. The consistency device 201 may detect the update of the first metadata MD1 made by the first host H1 so that the information about the first storage area SA1 is changed. In this case, the consistency device 201 may multicast the update information to any other hosts (e.g., H2, H3, and H4) sharing the first storage device 210 through the CXL switch SW_CXL, and the hosts H2, H3, and H4 may update the metadata based on the update information INF_u received through the CXL switch SW_CXL. Therefore, without a separate communication or protocol between the hosts H1 to H4, consistency can be maintained between the metadata managed by the hosts H1 to H4.

[0169] In an embodiment, the consistency device 201 may include a sharing table in which information about the sharing relationship between the plurality of hosts H1 to H4 and the plurality of storage devices 210 to 230 is stored. For example, the plurality of hosts H1 to H4 and the plurality of storage devices 210 to 230 may have a reference Figure 4 In this case, the consistency device 201 may include a reference Fig.12 The shared table ST-1 described above. The consistency device 201 can multicast the update information INF_u based on the shared table ST-1. In an embodiment, the structure (or architecture) of the storage system 200 can be changed so that a new host (or storage device) is added, an existing host (or storage device) is replaced with a new host (or storage device), or an existing host (or storage device) is removed. When those kinds of changes occur, the shared table ST-1 can be updated.

[0170] Fig.18 It's a picture. Fig.17Flow chart of the operation of the storage system 200. For ease of description, it is assumed that the first storage device 210 is shared by the first host H1, the second host H2, and the third host H3, and the first host H1 performs access to the first storage device 210 and updates information of the first metadata MD1 corresponding to the first storage area SA1 of the first storage device 210. However, the present invention is not limited thereto. For example, the storage system 200 may perform reference Figures 1 to 17 Describes various operations (e.g., data copy operations and data move operations).

[0171] refer to Fig.17 and Fig.18 , the first host H1, the CXL switch SW_CXL, and the first storage device 210 may perform operations S610 to S622. Operations S610 to S622 are similar to Fig. 9 Operations S210 to S231 of the present invention are described herein, and thus, repeated descriptions of the same, similar, or functionally equivalent elements (or operations) may not be given to avoid redundancy.

[0172] In operation S630, consistency device 201 may detect access request RQ_ACC through CXL switch SW_CXL. For example, consistency device 201 may be configured to monitor or listen to CXL switch SW_CXL. Consistency device 201 may detect access request RQ_ACC transmitted from first host H1 to first storage device 210 through CXL switch SW_CXL. Alternatively, consistency device 201 may detect access response REP_ACC transmitted from first storage device 210 to first host H1 through CXL switch SW_CXL.

[0173] In operation S631, the consistency device 201 may determine whether there is update information of the first metadata MD1 associated with the first storage area SA1 based on the access request RQ_ACC (or the access response REP_ACC). The configuration for determining whether there is update information of the first metadata MD1 associated with the first storage area SA1 is described above, and therefore, repeated descriptions of the same, similar, or functionally equivalent elements (or operations) may not be given to avoid redundancy.

[0174] When it is determined that no update of the first metadata MD1 is made, the consistency device 201 may not perform a separate operation.

[0175] When it is determined that the update of the first metadata MD1 is made, in operation S640, the consistency device 201 may send update information INF_u to the CXL switch SW_CXL. The update information INF_u may include information updated by the access request RQ_ACC in association with the first storage area SA1 of the first storage device 210. In operation S641, the CXL switch SW_CXL may transmit the update information INF_u to the second host H2. In operation S642, the CXL switch SW_CXL may transmit the update information INF_u to the third host H3.

[0176] In an embodiment, based on the sharing table, the consistency device 201 may send information (e.g., PID2 / cbp_H1, PID3 / cbp_H2, and PID4 / cbp_H3) about hosts (e.g., H1, H2, and H3) that share the first storage device 210 together with the update information INF_u to the CXL switch SW_CXL. The CXL switch SW_CXL may multicast the update information INF_u to the second host H2 and the third host H3 based on the above information.

[0177] In operation S651, the second host H2 may update the second metadata MD2 based on the update information INF_u. In operation S652, the third host H3 may update the third metadata MD3 based on the update information INF_u.

[0178] Fig.19 It's a picture. Fig.17 A flowchart of the operation of the storage system. Fig.17 and Fig.19 , the first host H1, the CXL switch SW_CXL, and the first storage device 210 may perform operations S710 to S722. Operations S710 to S722 are similar to Fig. 9 Operations S210 to S231 of the present invention are described herein, and thus, repeated descriptions of the same, similar, or functionally equivalent elements (or operations) may not be given to avoid redundancy.

[0179] In operation S730, the first storage device 210 may determine whether the first metadata MD1 is updated. Operation S730 is similar to Fig. 9 The first storage device 210 may not perform a separate operation when it is determined that no update of the first metadata MD1 is made.

[0180] When it is determined that the update of the first metadata MD1 is made, in operation S731, the first storage device 210 may send update information INF_u about the first metadata MD1 to the CXL switch SW_CXL. In an embodiment, the update information INF_u may include information of the first metadata MD1 corresponding to an area belonging to the first storage area SA1. In operation S732, the CXL switch SW_CXL may transmit the update information INF_u to the consistency device 201.

[0181] In operation S740, the consistency device 201 may search for a shared host based on the received update information INF_u. For example, when the update information INF_u includes information or an address associated with the first storage area SA1 of the first storage device 210, the consistency device 201 may search for hosts (e.g., H1, H2, and H3) that share the first storage device 210 based on the sharing table. Thereafter, the consistency device 201, the CXL switch SW_CXL, the second host H2, and the third host H3 may perform operations S751 to S762. Operations S751 to S762 are similar to those described in reference 1. Fig.18 Operations S640 to S652 are described, and thus, repeated descriptions of the same, similar, or functionally equivalent elements (or operations) may not be given to avoid redundancy.

[0182] As described above, according to an embodiment of the present invention, a plurality of hosts H1 to H4 and a plurality of storage devices 210 to 230 may communicate with each other through a CXL switch SW_CXL. A consistency device 201 may be connected to the CXL switch SW_CXL. Based on access requests sent to the plurality of storage devices 210 to 230 or update information provided from the plurality of storage devices 210 to 230, the consistency device 201 may detect whether metadata is updated by the plurality of hosts H1 to H4, and may multicast the update information to the plurality of hosts H1 to H4. The plurality of hosts H1 to H4 may update the corresponding metadata based on the update information. Therefore, consistency may be maintained between the plurality of metadata of the plurality of hosts H1, H2, H3, and H4 without separate communication or protocol between the plurality of hosts H1 to H4.

[0183] Fig. 20 is a block diagram illustrating the topology of a storage system according to an embodiment of the present invention. Fig. 20 , the storage system 1000 may include a client node 1001, a network NT, a plurality of hosts 1010 to 1030, a plurality of network interface controllers NIC1 to NIC3, a plurality of CXL switches SW_CXL1 to SW_CXL3, and a plurality of storage devices 1110 to 1320. In an embodiment, the storage system 1000 may be a decomposed storage system. For example, in Fig. 20In the illustrated configuration, new hosts are added, existing hosts are replaced with new hosts, or existing hosts are removed.

[0184] The client node 1001 may be connected to the network NT. The client node 1001 may be a system or computing node that requests various operations or functions from the storage system 1000 through the network NT. In an embodiment, the network NT may be implemented based on various protocols such as the following: TCP / IP network protocol, Fibre Channel (FC) protocol, Ethernet protocol, storage-specific network protocol (such as storage area network protocol), FC over Ethernet (FCoE) protocol, network attached storage (NAS) protocol, and fabric-based NVMe (NVMe-oF) protocol.

[0185] A plurality of network interface controllers NIC1 to NIC3 may be connected to a network NT. A plurality of network interface controllers NIC1 to NIC3 may be connected to a plurality of CXL switches SW_CXL1 to SW_CXL3, respectively. A plurality of network interface controllers NIC1 to NIC3 may be configured to provide an interface between the network NT and the plurality of CXL switches SW_CXL1 to SW_CXL3. In an embodiment, a plurality of network interface controllers NIC1 to NIC3 may be configured to support the PCIe protocol.

[0186] A plurality of hosts 1010, 1020, and 1030 may be connected to a plurality of CXL switches SW_CXL1 to SW_CXL3, respectively. Storage devices 1110 and 1120 may be connected to a first CXL switch SW_CXL1, storage devices 1210 and 1220 may be connected to a second CXL switch SW_CXL2, and storage devices 1310 and 1320 may be connected to a third CXL switch SW_CXL3. The plurality of CXL switches SW_CXL1 to SW_CXL3 may communicate with each other.

[0187] In an embodiment, multiple hosts 1010, 1020, and 1030 and client node 1001 may be reference nodes. Figures 1 to 19 The plurality of hosts H1 to H4 described above and the plurality of storage devices 1110 to 1320 may be reference Figures 1 to 19 The storage system 1000 can be based on the storage devices 110 to 130 described in the reference Figures 1 to 19 The described method of operation works.

[0188] Fig.21 is a diagram of a data center to which a storage device is applied according to an embodiment.

[0189] refer to Fig.21, the data center 2000 may be a facility that collects various types of data blocks and provides services, and is referred to as a data storage center. The data center 2000 may be a system for operating a search engine and a database, and may be a computing system used by a company such as a bank or a government agency. The data center 2000 may include application servers 2100 to 2100n and storage servers 2200 to 2200m. According to an embodiment, the number of application servers 2100 to 2100n and the number of storage servers 2200 to 2200m may be selected differently. The number of application servers 2100 to 2100n may be different from the number of storage servers 2200 to 2200m.

[0190] The application server 2100 or the storage server 2200 may include at least one of the processors 2110 and 2210 and the memories 2120 and 2220. The storage server 2200 will now be described as an example. The processor 2210 may control all operations of the storage server 2200, access the memory 2220, and run instructions and / or data loaded in the memory 2220. The memory 2220 may be a double data rate synchronous DRAM (DDR SDRAM), a high bandwidth memory (HBM), a hybrid memory cube (HMC), a dual in-line memory module (DIMM), an Optane DIMM, and / or a non-volatile DIMM (NVMDIMM). In some embodiments, the number of processors 2210 and the memory 2220 included in the storage server 2200 may be selected differently. In an embodiment, the processor 2210 and the memory 2220 may provide a processor-memory pair. In an embodiment, the number of processors 2210 may be different from the number of memories 2220. The processor 2210 may include a single-core processor or a multi-core processor. The above description of the storage server 2200 may be similarly applicable to the application server 2100. In some embodiments, the application server 2100 may not include the storage device 2150. The storage server 2200 may include at least one storage device 2250. According to embodiments, the number of storage devices 2250 included in the storage server 2200 may be selected differently.

[0191] Application servers 2100 to 2100n can communicate with storage servers 2200 to 2200m through network 2300. Network 2300 can be implemented by using Fibre Channel (FC) or Ethernet. In this case, FC can be a medium for relatively high-speed data transmission and uses an optical switch with high performance and high availability. Storage servers 2200 to 2200m can be provided as file storage, block storage or object storage according to the access method of network 2300.

[0192] In an embodiment, the network 2300 may be a storage-specific network, such as a storage area network (SAN). For example, the SAN may be an FC-SAN implemented using an FC network and according to an FC protocol (FCP). As another example, the SAN may be an Internet Protocol (IP)-SAN implemented using a transmission control protocol TCP / IP network and according to a SCSI or Internet SCSI (iSCSI) protocol based on TCP / IP. In another embodiment, the network 2300 may be a general-purpose network, such as a TCP / IP network. For example, the network 2300 may be implemented according to protocols such as FC over Ethernet (FCoE), network attached storage (NAS), and NVMe over fabric (NVMe-oF).

[0193] Hereinafter, the application server 2100 and the storage server 2200 will be mainly described. The description of the application server 2100 may be applicable to another application server 2100n, and the description of the storage server 2200 may be applicable to another storage server 2200m.

[0194] The application server 2100 may store data requested to be stored by a user or client in one of the storage servers 2200 to 2200m through the network 2300. Likewise, the application server 2100 may obtain data requested to be read by a user or client from one of the storage servers 2200 to 2200m through the network 2300. For example, the application server 2100 may be implemented as a web server or a database management system (DBMS).

[0195] The application server 2100 may access a memory 2120n or a storage device 2150n included in another application server 2100n through the network 2300. Alternatively, the application server 2100 may access a memory 2220 to 2220m or a storage device 2250 to 2250m included in the storage servers 2200 to 2200m through the network 2300. Therefore, the application server 2100 may perform various operations on the data stored in the application servers 2100 to 2100n and / or the storage servers 2200 to 2200m. For example, the application server 2100 may execute an instruction for moving or copying data between the application servers 2100 to 2100n and / or the storage servers 2200 to 2200m. In this case, data may be moved from the storage devices 2250 to 2250m of the storage servers 2200 to 2200m to the storages 2120 to 2120n of the application servers 2100 to 2100n directly or through the storages 2220 to 2220m of the storage servers 2200 to 2200m. The data moved through the network 2300 may be data encrypted for security or privacy.

[0196] The storage server 2200 will now be described as an example. The interface (I / F) 2254 may provide a physical connection between the processor 2210 and the controller (CTRL) 2251 and a physical connection between the network interface (NI) 2240 and the controller 2251. For example, the interface 2254 may be implemented using a direct attached storage (DAS) solution in which the storage device 2250 is directly connected to a dedicated cable. For example, the interface 2254 may be implemented using various interface solutions such as ATA, SATA, e-SATA, SCSI, SAS, PCI, PCIe, NVMe, IEEE 1394, USB interface, SD card interface, MMC interface, eMMC interface, UFS interface, eUFS interface, and / or CF card interface.

[0197] The storage server 2200 may further include a switch 2230 and a NI 2240. The switch 2230 may selectively connect the processor 2210 to the storage device 2250 or selectively connect the NI 2240 to the storage device 2250 via the control of the processor 2210.

[0198] In an embodiment, the NI 2240 may include a network interface card and a network adapter. The NI 2240 may be connected to the network 2300 through a wired interface, a wireless interface, a Bluetooth interface, or an optical interface. The NI 2240 may include an internal memory, a digital signal processor (DSP), and a host bus interface, and is connected to the processor 2210 and / or the switch 2230 through the host bus interface. The host bus interface may be implemented as one of the above examples of the interface 2254. In an embodiment, the NI 2240 may be integrated with at least one of the processor 2210, the switch 2230, and the storage device 2250.

[0199] In the storage servers 2200 to 2200m or the application servers 2100 to 2100n, the processor may send a command to the storage devices 2150 to 2150n and 2250 to 2250m or the memories 2120 to 2120n and 2220 to 2220m and program or read the data. In this case, the data may be data in which an error is corrected by an ECC engine. The data may be data on which a data bus inversion (DBI) operation or a data masking (DM) operation is performed, and may include cyclic redundancy code (CRC) information. The data may be data encrypted for security or privacy.

[0200] The storage devices 2150 to 2150n and 2250 to 2250m may send control signals and command / address signals to the NAND flash memory devices 2252 to 2252m in response to a read command received from the processor. Therefore, when data is read from the NAND flash memory devices 2252 to 2252m, a read enable (RE) signal may be input as a data output control signal, and thus, data may be output to the DQ bus. The RE signal may be used to generate a data strobe signal DQS. Depending on the rising edge or falling edge of the write enable (WE) signal, the command and address signals may be latched in the page buffer.

[0201] The controller 2251 may control all operations of the storage device 2250. In an embodiment, the controller 2251 may include an SRAM. The controller 2251 may write data to the NAND flash memory device 2252 in response to a write command or read data from the NAND flash memory device 2252 in response to a read command. For example, the write command and / or the read command may be provided from the processor 2210 of the storage server 2200, the processor 2210m of another storage server 2200m, or the processors 2110 and 2110n of the application servers 2100 and 2100n. The DRAM 2253 may temporarily store (or buffer) data to be written to the NAND flash memory device 2252 or data read from the NAND flash memory device 2252. Likewise, the DRAM 2253 may store metadata. Here, the metadata may be user data or data generated by the controller 2251 to manage the NAND flash memory device 2252. The storage device 2250 may include a secure element (SE) for security or privacy.

[0202] In an embodiment, the network 2300 may be used with reference Figures 1 to 20 In this case, the application servers 2100 to 2100n and the storage servers 2200 to 2200m can be based on the reference Figures 1 to 20 Alternatively, the application servers 2100 to 2100n and the storage servers 2200 to 2200m may constitute a reference Figures 1 to 20 The storage system described (i.e., the distributed storage system) and can be based on the reference Figures 1 to 20 The described method of operation works.

[0203] According to the present invention, in a decomposed storage system, a storage device can detect whether metadata of a host has changed, and can send update information corresponding to the change in metadata to any other host. Each other host can update its metadata based on the update information received from the storage device. Therefore, multiple hosts can maintain consistency of metadata about a storage device shared by multiple hosts without separate communication.

[0204] While the present invention has been described with reference to the embodiments thereof, it will be apparent to those skilled in the art that various changes and modifications can be made thereto without departing from the spirit and scope of the invention as set forth in the following claims.

Claims

1. A method for operating a storage device connected to a computing fast link switch, the method comprising: receiving a first access request from a first host via the computing fast link switch; Execute an operation corresponding to the first access request; detecting, based on the first access request, a change in first metadata managed by the first host; as well as First update information corresponding to the change in the first metadata is sent to a second host sharing the storage device through the computing fast link switch.

2. The method according to claim 1, in, The first access request includes a first address corresponding to a storage area of ​​the storage device, and The first update information includes the first address.

3. The method according to claim 2, in, The first access request is based on a CXL.io protocol which is a non-coherent input / output protocol based on PCI-e, and The first address is a logical block address managed by the file system of the first host.

4. The method according to claim 2, in, The first access request is based on a CXL.mem protocol as a memory access protocol supporting memory access, and The first address is a memory physical address indicating an area of ​​a system memory managed by the first host.

5. The method according to claim 1, in, The first host manages a storage area of ​​the storage device based on the first metadata, and The second host manages the storage area of ​​the storage device based on second metadata.

6. The method according to claim 5, wherein: The second host updates the second metadata based on the first update information.

7. The method according to claim 1, further comprising: The first update information is sent to a third host sharing the storage device through the computing fast link switch.

8. The method according to claim 1, further comprising: receiving a second access request from the second host via the computing fast link switch; executing an operation corresponding to the second access request; detecting, based on the second access request, a change in second metadata managed by the second host; as well as Second update information corresponding to the change in the second metadata is sent to the first host sharing the storage device through the computing fast link switch.

9. The method according to claim 1, wherein: Executing an operation corresponding to the first access request includes: First data stored in the storage device is sent to an external storage device through the computing fast link switch in response to the first access request.

10. A storage device, comprising: Non-volatile memory devices; Dynamic random access memory devices; as well as a controller configured to control the nonvolatile memory device and the dynamic random access memory device in response to a first access request received from a first host among a plurality of hosts, Wherein, the controller comprises: a computing fast link interface circuit connected to a computing fast link switch and configured to receive the first access request from the first host through the computing fast link switch; a NAND interface circuit configured to control the nonvolatile memory device based on the first access request; a dynamic random access memory controller configured to control the dynamic random access memory device based on the first access request; and A consistency circuit configured to detect a change in first metadata managed by the first host in response to the first access request, and to send first update information corresponding to the change in the first metadata to other hosts among the plurality of hosts that share the storage device through the compute fast link switch.

11. The storage device according to claim 10, in, The coherence circuit includes a shared table, and The sharing table includes a port identifier and a callback parameter of each of the other hosts that share the storage device.

12. The storage device according to claim 11, wherein: The coherence circuit sends the first update information to each of the other hosts based on the port identifier and the callback parameter of the shared table.

13. The storage device according to claim 10, in, The first access request includes a first address corresponding to a storage area of ​​the storage device, and The first update information includes the first address.

14. The storage device according to claim 13, in, The first access request is based on a CXL.io protocol which is a non-coherent input / output protocol based on PCI-e, and The first address is a logical block address managed by the file system of the first host.

15. The storage device according to claim 13, in, The first access request is based on a CXL.mem protocol as a memory access protocol supporting memory access, and The first address is a memory physical address indicating an area of ​​a system memory managed by the first host.

16. A method for operating a storage system, the method comprising: A storage system is provided, the storage system comprising: Calculate fast link switches; a first storage device connected to the computing fast link switch; a first host connected to the computing fast link switch; and a second host connected to the computing fast link switch, The first host sends a first access request corresponding to a storage area of ​​the first storage device to the first storage device through the computing fast link switch and updates first metadata; The first storage device receives the first access request through the computing fast link switch and performs an operation corresponding to the first access request; detecting, by the first storage device, a change in the first metadata and sending, through the compute fast link switch, first update information corresponding to the change in the first metadata to the second host; and receiving, by the second host through the computing fast link switch, the first update information and updating second metadata based on the first update information, The first host and the second host share the first storage device. wherein the first host manages the first storage area of ​​the first storage device based on the first metadata, and The second host manages the first storage area of ​​the first storage device based on the second metadata.

17. The method according to claim 16, wherein: The first access request is based on a CXL.io protocol that is a non-coherent input / output protocol based on PCI-e or a CXL.mem protocol that is a memory access protocol supporting memory access.

18. The method according to claim 16, in, The storage system further includes a second storage device connected to the computing fast link switch, and The first storage device executing the operation corresponding to the first access request includes: sending the first data stored in the first storage device to the second storage device through the computing fast link switch.

19. The method according to claim 18, further comprising: storing, by the second storage device, the first data received from the first storage device in a second storage area of ​​the second storage device; detecting, by the second storage device, a change in the first metadata and sending, via the compute fast link switch, second update information corresponding to the change in the first metadata to the second host; as well as updating the second metadata by the second host based on the second update information, The first host and the second host share the second storage device. wherein the first host further manages the second storage area of ​​the second storage device based on the first metadata, and The second host further manages the second storage area of ​​the second storage device based on the second metadata.

20. The method according to claim 16, wherein: The storage system is a disaggregated storage system.

Images